MLS EDUCATIONAL RESEARCHwww.mlsjournals.com/ISSN: 2603-5820 |
|
Educational Research Journal 2018, Vol 1 n. 1;
How to cite this article:
Rivera Vélez, R. C. (2018). Study of Undergraduate Nursing Students’ Perception on the Objective Structured Clinical Evaluation (OSCE) when used in Clinical Simulations in Nursing Courses in Puerto Rico. Educational Research, 1 (1), _-_. Doi:____
Abstract. This study aims to explore a representative sample of undergraduate nursing students’ perception on the use of the Objective Structured Clinical Evaluation (OSCE / ECOE) in nursing courses in a university of Puerto Rico. The OSCE is an educational methodology internationally recognized for its validity and reliability when evaluating clinical competencies in professionals within the field of health sciences in a formative and (or) summative manner (Harden, 2016; Abdulghani, Ponnamperuma & Amin, 2015). This study follows a quantitative, nonexperimental, descriptive, transversal design. Information was gathered using a semi-structured questionnaire with open-ended questions and closed-ended items using a Likert scale. Central tendency, dispersion, frequency, percentages and Pearson’s correlation coefficient were used for statistical analysis. Open-ended questions related to strengths, limitations and recommendations on the OSCE mentioned by students were examined using content analysis. The percentage and absolute distribution of students per item for the questionnaire used in this study revealed that participants perceived the OSCE as a helpful tool in the evaluation of nursing skills. Moreover, the results proved the effectiveness of OSCE for measuring the achievement of professional competencies in nursing courses when considering the students’ input. Using the students’ views regarding the strengths and limitations of the OSCE, room for improvement was identified upon using it in nursing courses.
Keywords: Perception, OSCE, competence, nursing students, simulation
Resumen. El estudio que se presenta estuvo dirigido a la exploración de la percepción de estudiantes de Bachillerato en Enfermería sobre la utilización de la Evaluación Clínica Objetiva Estructurada (ECOE / OSCE) en cursos de enfermería en una universidad en Puerto Rico. La ECOE, es una metodología educativa internacionalmente reconocida por su validez y fiabilidad para evaluar las competencias clínicas en los profesionales de ciencias de la salud de manera formativa y (o) sumativa. El paradigma de la investigación es cuantitativo no experimental descriptivo transversal. Los datos fueron recopilados, mediante la aplicación de un cuestionario semiestructurado con preguntas abiertas y cerradas utilizando la escala Likert. Para el análisis estadístico se utilizaron medidas de tendencia central y dispersión, frecuencias, porcentajes y el coeficiente de correlación de Pearson. Preguntas abiertas relacionadas con fortalezas, debilidades y recomendaciones relacionadas a la ECOE mencionadas por los estudiantes, fueron examinadas mediante el análisis de contenido. Los resultados de la distribución porcentual y absoluta de los estudiantes por ítems del cuestionario utilizado para este estudio, revelaron que los participantes percibían la ECOE como una herramienta de ayuda en la evaluación de destrezas clínicas en enfermería. Los resultados comprobaron, además, la efectividad de la ECOE para medir el logro de las competencias profesionales en cursos de enfermería. Mediante las propias expresiones de los estudiantes en cuanto a las fortalezas y debilidades de la ECOE, pudieron identificarse áreas a mejorar al utilizar la misma en cursos de enfermería.
Palabras claves: Percepción, ECOE, competencias, estudiantes de Enfermería, simulación
Justification and Problem
Studies have found that newly graduated nurses do not have the clinical skills required to provide high-quality care to their patients (Hengstherger-Sims et al., 2008). Nursing educators have the responsibility of ensuring that their students achieve the required nursing-related competencies. (Stoll, 2015; Jeffries, 2012). According to Standard 4.7 of the Accreditation Commission for Education in Nursing (ACEN) in 2015, evaluation methodologies must be varied, reflect established professional and practice competencies, and measure the achievement of the nursing student’s learning outcomes.
With the challenges of providing high-quality clinical experiences, nursing education is at a crossroad between tradition and innovation. Traditional assessment methods have a different approach to the integration of clinical competency examinations. It is therefore necessary to diversify the curriculum by integrating evaluative methodologies with different approaches and to provide proof of validity and reliability in the evaluation of clinical skills. The OSCE is an alternative for achieving this (Khattab & Rawlings, 2008) and can provide more evidence of learning outcomes and professional competency achievement.
According to Oranye, et al. (2014), the OSCE should be included in the design of nursing programs and in the assessment of professional nursing competence level. The integration of the OSCE in nursing programs could help prepare the students so that they can successfully carry out the required assessments in order to be hired in different parts of the world (Merrifield, 2016). If the OSCE can be a determining or promoting factor in a student’s success within their study program, there must be valid proof or research evidence related to the use of the OSCE.
The aim of this study is to know what undergraduate nursing students think about the incorporation and use of the OSCE at the end of their nursing courses at a university in Puerto Rico. It is also expected to study the achievement of the competencies attained by the participants using this assessment method. The findings from this research may help to integrate the OSCE into nursing programs in Puerto Rico. It may also be an opportunity to share knowledge about the best practices in the use of this assessment modality within a clinical simulation environment from students’ perspectives.
General Objective
To know the opinion of undergraduate nursing students about the Objective Structured Clinical Evaluation (OSCE) in a clinical simulation for nursing courses at a university in Puerto Rico.
Hypothesis
H1 Students view the OSCE as a tool to assist in the assessment of clinical and non-clinical nursing skills.
H2 The OSCE facilitates the identification of the professional competence level achieved by students at the end of a nursing course.
Literature Review
Assessment measures quality and productivity to achieve a standardized performance level (Bourke & Ihrke, 2005) and represents the evaluation of learning outcomes. It tells us how capable the student is about meeting certain skills, knowledge and attitudes to solve problems within the field of health. It is important to remember that what is not evaluated is devalued (Goñi, 2005). If there is assessment, there is regulation. Through regulation, the student knows the areas in need of improvement and can stop making the same mistakes, which keep them from achieving the required learning outcomes and competencies.
In 1975 Dr. Ronald McGregor Harden published a new method of estimating clinical competencies, the Objective Structured Clinical Examination (OSCE). The OSCE was applied to medical students in Scotland in order to resolve some of the difficulties identified in traditional assessment methods. Since 1979, the timed examination has been modified, developed and officially named the “OSCE”, becoming a pillar of worldwide clinical assessment (Harden & Gleeson, 1979; Abdulghani, Ponnamperuma & Amin, 2015). The OSCE was originally described as “a timed examination” in which the medical student interacted with a series of simulated patients in 16 stations, while an examiner observed and evaluated their clinical performance with a checklist.
At each OSCE station, the student could perform different challenges within 5 minutes. At the end of each stations the student would then move to subsequent stations with questions related to the standardized patients from the previous ones. This included performing a medical history and physical examination, the counseling and treatment of the patient. The use of the OSCE in nursing was integrated for the first time at the University of Ottawa in Canada. (Ross et al., 1988). The study from this experience suggested that the OSCE could be a powerful tool in the evaluation of clinical competencies. According to Ross, et al. (1988), it could also be an effective facilitator for learning to perform clinical skills in nursing, as well as in the field of medicine.
The OSCE makes it easier to assess the parallelism between clinical performances in simulated situations within the clinical laboratory and actual clinical practice situations. It is a form of estimation that examines clinical competencies, such as medical history, physical examination, communication skills, ethics, attitudes, and professionalism (Harden et al., 1975). It is characterized by a high level of planning, coordination, and structuring of scenarios in a simulated clinical environment that evaluates the competency components of an academic course or program (Caballero, 2012). The structure and design of the OSCE traditionally focused on the clinical objectives corresponding to the simulated experiences practiced and are evaluated in checklist formats (Caballero et al., 2012). The OSCE is used to evaluate the clinical practice in many disciplines in the United States and usually consists of 12 stations of 5-15 minutes each. The total OSCE process concludes in 8 hours with 2 breaks. In the simulated clinical environment in nursing, the evaluation facilitates objective, controlled and safe estimation of clinical and non-clinical health care skills.
In a pilot study called “NURSEOSCE”, the application of the OSCE was evaluated in a formative and summative way at a nursing school in the Universidad de Carolina del Este (University of East Carolina). The faculty, students, and standardized patients found that the OSCE was a worthwhile educational experience. The students’ response was positive, and they recognized the good preparation obtained through the OSCE at the end of the clinical experience. Some students suggested that the OSCE should be a mandatory assessment (Rentschler, Eaton, Cappiello, McNally & McWilliam, 2007). Several advanced nursing programs have already incorporated an OSCE format. One of these formats used in an Advanced Practice Nursing Graduate program includes the estimation of skills by incorporating volunteers who act as standardized patients. (Khattab & Rawlings, 2008).
According to Walsh et al, 2009, the OSCE is an efficient way of assessing skills and knowledge by rotating students through stations where a clinical environemnt is simulated. The number of scenarios can vary from 4 to 42, with 4 to 15 minutes for each station. Another study found that the OSCE improves skills and knowledge in calculus and medical drug administration (Meechan, Jones & Valler-Jones, 2011). In another qualitative descriptive research study, midwifery students perceived greater effectiveness in assessing obstetric emergencies and preparing students for clinical practice through OSCE compared to other forms of evaluation. They recognized the OSCE as an excellent tool for increasing depth in learning (Barry et al., 2012) and for developing critical thinking.
At the St. Sebastian University in Chile, they considered it necessary to incorporate the OSCE into the nursing career. During the investigation of the students’ perception of this methodology, they perceived the OSCE as a positive evaluation experience. In addition, they recognized it as an opportunity to strengthen learning. Some felt that seven minutes was insufficient at some stations. In addition, they found it stressful, but only at the beginning, due to their lack of experience with this evaluation method (Alarcón A., 2013).
In 2013, the University of Cadiz (UC) started the innovative integration of the Objective Structured Evaluation of Nursing Care (OSENC) evaluation methodology based on the OSCE. The same methodology is used in the Practicum courses of the Department of Nursing and Physiotherapy (Moreno Coral, 2013). The UC has produced specific guides describing the environment and conditions that facilitate the effectiveness of the OSENC. This university planned the design of the stations based on the activities listed in a Portfolio.
In a research conducted in 2014 in Australia, they developed and integrated guidelines for best OSCE practices in nursing. The study suggests that the integration of best practice guidelines into OSCE is an effective tool for assessing the learning of nursing students’ knowledge and skills (Kelly et al., 2016). Some hospitals in Canada currently require nurses to pass an OSCE examination for recruitment (Merrifield, 2016). There is no specific design or structure to carry out the OSCE, each institution develops their own (Scalabrini, 2015). The institution participating in this research designed the OSCE based on its curricular program and its characteristics according to the literature. It considers the principles established by Harden & Gleeson (1979) and the findings and recommendations of research carried out on their incorporation into the different disciplines and nursing programs from different countries.
The design of this research corresponds to the paradigm of cross-sectional nonexperimental descriptive quantitative research. The variables sampled in this study were professional competence, OSCE and perception.
Population and Sample
The population of this study was composed of students from two undergraduate degree courses in nursing who participated in an OSCE at a university in Puerto Rico. The sample size was determined by the institution that authorized the sample considering the specific number of students who participated in an OSCE during the time of the research. The institution in Puerto Rico that granted the opportunity to conduct the study, authorized the participation of up to a maximum of l05 students in the courses that were being evaluated through an OSCE during a specific academic semester. It was possible to obtain 100% of the sample allowed for this study.
Instruments
An instrument designed, validated and submitted to expert opinions in a research with similar characteristics regarding this study, and conducted by Dr. Ángela Alarcón (2015) at a university in Chile, was used with suitable authorizations and cultural adaptations. The instrument or questionnaire finally used in this study was design in order to evaluate undergraduate nursery students’ perceptions about the OSCE test in nursing courses at a university in Puerto Rico. The internal consistence in the questionnaire was assessed trough the Cronbach alpha formula. Significance level was set at p <0.05.
A test run of the questionnaire was made with 77 students with similar characteristics to the sample chosen for the formal study. The instrument largely meets the basic case that, if a factorial analysis can be carried out, the recommended number of subjects involved must be between two and ten times the number of items. Once the sample size was confirmed to be adequate for studying the instrument’s technical quality, validity evidences in the scale were analyzed.
As a prelude to the construct validity analysis, the homogeneity of the questionnaire was calculated and any item with a low discrimination level and, therefore, a <200 correlation with the total in the scale. In this study, the questionnaire’s application for gathering all necessary data was possible after obtaining the required permissions. Before its application, these permissions were granted at the university where the study was carried out and by the subjects themselves. The instrument designed for this research includes 17 closed-ended questions and 4 open-ended questions.
In the first section, the general and sociodemographic aspects of the participants in the study were identified. In addition, an open-ended question was included to know what the students expected their final scores would be in the OSCE test. In the second section, specific or close-ended questions for getting the students’ opinion about the OSCE test were presented, using as quantitative analysis: (5) I completely agree, (4) I agree, (3) Indecisive, (2) I disagree, (1) I completely disagree. Items 1 to 10 of the Perception Scale were created for the research on the perceptions about the OSCE as an assessment tool and the students’ opinions when participating on the assessment.
-
The OSCE enables assessing:
- Theoretical knowledge
- Clinical skills
- Ethical behavior
- Communication skills
- Stations are realistic and representative of real-world clinical practice
- The OSCE measures the established objectives
- The stations reflect the contents studied during the course
- Would you consider the OSCE a stressful assessment
- Were the allotted times enough for completing the stations
- The presence of an observer, in the stations, interferes with your work
- The OSCE is beneficial for professional training
- Would you consider the OSCE an effective learning experience
- OSCE must be included as assessment method in this course
In the third section, three open-ended questions were made in which students could mention the strength and weak points of the OSCE test as an assessment strategy, as well as any recommendations for improving it.
Analysis and Results Discussion
The SPSS 19 statistic program from IBM was used to analyze the data. Statistical analysis was based on a descriptive analysis through absolute distributions (frequencies), average result and percentile distributions (qualitative variables). In addition, the central tendency and spread measures were calculated.
A Student t-Test was conducted in order to compare the average values (arithmetic mean) between the students’ expected scores and the obtained scores. This deepened the identification of the level of competences reached by the students. Pearson’s correlation coefficient is used in the explanation. A multivariate analysis was carried out in order to identify possible confounding variables..
Open-ended questions related to the strengths, weaknesses and recommendations mentioned by the students were examined through content analysis. This research technique reveals meanings and assessments through essentially verbal or communicative data shared by the participants. It uses the recording, coding, and summary of expressions provided in categorizations that allow for efficient representations that simplify the collected volume of data (Krippendorff, 2013).
Respondents had the opportunity to express their opinions on the OSCE after being evaluated in the achievement of competencies at the end of two nursing courses. Their expressions helped to identify, based on evidence, that the incorporation of this assessment method can help to identify the levels of competencies attained by students in their curriculum. It was possible to identify the strengths, weaknesses and recommendations to improve the use of the OSCE from the students’ perspective.
The socio-demographic profile of the participants in this study was constituted by a sample of 105 undergraduate nursing students from a university in Puerto Rico. It was possible to obtain 100% of the sample allowed for this study. It was made up of students enrolled in the Adult Nursing Care II course (Third year) and in the Integrated Clinical Nursing Practice course (Fourth year) of the Nursing undergraduate program.
El 68.6% de los estudiantes encuestados (n = 72) participó en la ECOE en el curso del tercer nivel. Por su parte, el 31.4% de los estudiantes (n = 33) participó en el curso del cuarto nivel. La muestra de estudiantes que participó de este estudio, en su mayoría (73.3%), pertenecía al sexo femenino. La figura 1 resume esta distribución por sexo. Hubo representación en todos los grupos de edad, siendo el mayor porciento de estudiantes participantes entre los 21 a 30 años.
In order to validate the effectiveness of the OSCE in measuring the achievement of professional competencies of nursing students in simulated clinical scenarios, a factorial analysis of main components was performed.
This factorial analysis allowed grouping the reagents in factors or components that could explain the variance observed in the subjects’ answers. The analysis of the main components and the Varimax rotation showed the convergence of two factors. Under the first factor, seven items that allude to Perception in terms of the OSCE as an assessment tool were gathered. On the other hand, the second factor was composed by the six remaining items that evaluate the perception on participating in the OSCE.
Figure 1.Nursing students participating in the study by gender
The items were configured according to the degree of saturation in each factor. All items had a factorial load greater than 0.3. Factor 1, perception of the OSCE as an assessment tool, reveal items with a factorial load between 0.393 and 0.820. This factor encompasses items that emphasize that the OSCE allows for the evaluation of theoretical knowledge, clinical skills, ethical behavior, and communication skills, as well as the realism, objectives, and content of the OSCE and its stations. As for Factor 2, perception of the experience of participation in the OSCE, factorial loads from 0.332 to 0.743 were observed. The items under this factor allude to aspects that can make the OSCE a beneficial experience for students, which is a stressful evaluation possibly due to the allotted times or the presence of an observer, and the desire to include it as an evaluation method.
As a second objective, the aim was to identify the level of competences attained by the students. The questionnaire included a question that auscultated the final score that the students participating in the OSCE expected to obtain. In addition to the information on the official grades of the institution, the score that the students finally obtained in the evaluation was extracted. The analysis of the results indicated that the level of proficiency achieved by the students was generally high.
The data corresponding to the expected score and the one finally obtained by the nursing students are shown in Figure 2. The students’ perception of the expected OSCE score tended to be 80-89%, by bringing 59% of the participants together. Among the scores finally obtained, the same line was the highest, but with 32.4%. The greater percentile of students, both in the expected and earned scores, were at the highest end of the scale (80-89% and 90-100%). In the score obtained, a notable percentage of students scoring less than 70% was observed, compared to the expected score (19% vs. 2.9%). In the score obtained, a notable percentage of students scoring less than 70% was observed, compared to the expected score (19% vs. 2.9%). The minimum number of students expected to get a final score of A or less than C.
Los datos correspondientes a la puntuación esperada y aquella finalmente obtenida por los estudiantes de enfermería se reseñan en el figura 2. Despunta que la percepción de los estudiantes con respecto a la puntuación esperada en la ECOE tendía a 80-89%, al reunir 59% de los participantes. Entre las puntuaciones finalmente obtenidas ese mismo renglón resultó ser el más alto, pero con 32.4%. Los porcentajes mayoritarios de estudiantes, tanto en puntuación esperada como obtenida, se ubicaron en el extremo que reunía a las calificaciones más altas (80-89% y 90-100%). En la puntuación obtenida se observó un porcentaje notable de estudiantes calificados menor de 70%, en comparación con la puntuación esperada (19% vs 2.9%). El mayor por ciento de los estudiantes esperaba obtener una puntuación final entre 80-89% (B). El mínimo de los estudiantes esperaba obtener una puntuación final de A o menor de C.
Estudio sobre la percepción de los estudiantes de bachillerato en enfermería al utilizar la evaluación clínica objetiva estructurada (ECOE) en simulación clínica en cursos de enfermería en Puerto Rico
Figure 2.Percentile distribution of expected and obtained final scores by students participating in the Objective Structured Clinical Evaluation (OSCE) within two nursing courses from a university in Puerto Rico (n = 105).
In order to compare the expected scores and those finally obtained, the Student t Test was used for comparing the means. The aim was to determine whether there were differences between the scores and whether they were significant. The mean scores were very similar. t Test results for related variables show a non-significant value (p = .491). The scores turned out to be correlated with each other, although the pass rate expected by the students was generally higher than the one finally obtained as a grade.
When the multivariate analysis of logistic regression was performed, we did not find a high degree of significance (p<0.05) of the sociodemographic variables of sex, age and the academic period in which they took the course to determine or predict the score finally obtained. With the specific third objective of collecting data related to the nursing students’ perception of the OSCE by means of questionnaire, the items comprising the questionnaire were analyzed. For this purpose, the classification of the items into the two factors that emerged from the analysis factor was taken into consideration. These are: Perception Factor 1 regarding the OSCE as an evaluation tool and Perception Factor 2 regarding the experience of participating in the OSCE.
The perception shared by the students through their responses in the questionnaire reflected a high valuation of the OSCE as a method of evaluation and formative experience. The highest percentage of students expressed strong agreement that the OSCE is a tool for assessing theoretical knowledge, clinical skills, ethical behavior and communication in nursing courses. The students participating in this study expressed strong agreement that the OSCE stations were realistic or representative of actual practice and that they were related to the content seen in the courses. They also expressed great agreement that the OSCE allowed measuring the objectives established in its design.
This first factor, OSCE as an evaluation tool, included seven items. Figure 3 summarizes their analysis. More than 50% of nursing students strongly agreed with these statements which referred to the positive aspects of the OSCE. It is clear from the figure that practically all students agreed or strongly agreed with the items. However, a significant percentage of them disagreed or were uncertain about the item that referred to the stations as realistic; representative of actual clinical practice. When ranking the items by percentage, it was found that more than 80% of nursing students strongly agreed that the OSCE allows the evaluation of clinical skills (81.9%) and ethical behavior (81%). Other high percentages in the very agreeable alternative were recorded in items that indicated that the OSCE allows evaluating theoretical knowledge (78.1%) and communication skills (76.2%).
These results denote the nursing students’ perception with respect to the evaluative value that the OSCE has for their professional training process. It also highlighted, with similar percentages, that the majority of the participating students perceived that the stations are in agreement with the course contents (73.3%) and that the OSCE measures the established objectives (71.4%). In turn, 57.1% of the students strongly agreed that the stations are realistic; that is, that they perceive them as representative of real clinical practice. This percentage, although majoritarian, turned out to be the lowest in the list of Factor 1 items. It is important to consider increasing the realism of the stations for improvement of the OSCE.
The nursing students’ perception about their experience of participation in the OSCE (Factor 2) was highly positive in terms of its benefits. However, a significant amount of them expressed that the allocated time and stress was a challenge in completing the stations. The highest percentage of students expressed disagreement in that the presence of an observer interferes with completing their work at the OSCE stations.
Figure 3.Distribution of students participating in the Objective Structured Clinical Evaluation (OSCE) by Factor 1; Perception based on OSCE as an evaluation tool.
We can see in Figure 4 that when organizing the items of Factor 2 by percentage achieved, about 70% or more of the nursing students mentioned strongly agreeing that the OSCE is an effective learning experience (74.3%), and that it should be included as an evaluation method in the course (72.4%). Another high percentage in the strongly agree choice was found in the item stating that the OSCE is beneficial for professional training (69.5%). Half of the participants (51.4%) did not consider the OSCE to be a stressful assessment. The results show that a considerable number of students perceived it as a stressful experience. This should be considered during the planning and implementation of the OSCE. In general terms, the results of the Factor 2 items denote the positive perception of nursing students regarding participation in this type of assessment.
A fundamental part of the study was to explore the nursing students’ perception regarding the strengths, needs and recommendations they identified when using the OSCE. These findings pointed to identifying aspects that were beneficial for students when participating in the OSCE, and areas of opportunity to give continuity to this tool as an evaluation method when training nursing professionals. Ninety-three (n=93) out of 105 nursing students who participated in the OSCE identified strengths in using this type of assessment.
These students provided 96 answers that were classified into 10 strengths, as shown in Table 1. The three most mentioned strengths by the participants were that the OSCE was beneficial for practice and professional training (33.3%); that it was a good self-evaluation strategy (25.8%); and highlighted its organization and structure (18.3%). An expression that very well characterized the strength of the OSCE as an effective self-evaluation strategy was the following report of a student: “Thanks to the OSCE, I was able to see my weaknesses and that I need to dedicate more time and effort in order to get good results and to perfect my skills”.
Figure 4. Distribution of students participating in the Objective Structured Clinical Evaluation (OSCE) by Factor 2, Perception on the participation experience within the OSCE
Table 1
Strengths that the participating students discovered when using the Objective Structured Clinical Evaluation (OSCE) in nursing
| Strengths | Frequency | Percentile |
|---|---|---|
| Benefit during practice and professional training | 31 | 33.3 |
| Good self-evaluation strategy | 24 | 25.8 |
| OSCE organization and structure | 17 | 18.3 |
| Stages as real and practical scenarios | 7 | 7.6 |
| Contributes to the development of skills | 4 | 4.3 |
| Develops self-confidence | 4 | 4.3 |
| Efficient learning experience | 3 | 3.2 |
| Human resources (professors, observers) | 3 | 3.2 |
| Time management | 2 | 2.2 |
| Materials | 1 | 1.1 |
In terms of its strength as a good self-assessment strategy, students agreed that the OSCE truly tests students, helps clarify doubts, and is an effective tool for practicing skills that could not be practiced in clinical areas. In addition, they mentioned that it helps to assess capacities to intervene in real (critical) stressful scenarios and enables them to identify areas for improvement before graduation, or whether they are qualified for what they have been trained for.
Seventy-six (n=76) out of 105 nursing students who participated in the OSCE identified needs or weaknesses of this assessment method. This group of participants provided 79 answers that were classified into 12 areas of need; that is, aspects that require attention. The list of requirements is summarized in Table 2. The need or weakness that can be considered the main one, as it is the most mentioned one by the participants, was the time factor required by the OSCE. 55.3% of the nursing students who participated in the OSCE alluded to time as a weakness of their experience with this assessment.
Table 2
Improvements that the participating students identified when using the Objective Structured Clinical Evaluation (OSCE) in nursing
| Weaknesses or improvements | Frequency | Percentile |
|---|---|---|
| Time factor | 42 | 55.3 |
| Creates stress | 12 | 15.8 |
| Materials | 7 | 9.2 |
| Need for more practice | 6 | 7.9 |
| Human resources | 4 | 5.3 |
| Previous notification | 2 | 2.6 |
| Communication | 1 | 1.3 |
| Evaluation in hospital or place of practice | 1 | 1.3 |
| Frequency | 1 | 1.3 |
| Late implementation | 1 | 1.3 |
| Student insecurity | 1 | 1.3 |
| Skill subdivision | 1 | 1.3 |
The stress-generating aspect (15.8%) and the materials used (9.2%) were also identified as areas of need for attention. Among the remaining answers provided, one group of students expressed the need for more practice (7.9%).
Eighty-seven (n =87) of the nursing students who participated in the OSCE identified recommendations regarding the use of the OSCE. These students provided 111 answers that were classified into 17 recommendations, which are outlined in Table 3. In addition, they made specific mentions aimed at the implementation of the OSCE (16%). These may include: that the OSCE continues to be administered more frequently or in each semester; that it be used in all or other courses, such as pediatrics, maternity and psychiatry;
that it be included as part of the record; that it not only be performed in the previous year, as it can help to reinforce the techniques taught. Other recommendations were aimed at conducting pre-course evaluations or tutoring before the process, improving the availability of equipment and materials, improving practice areas and stations, as well as facilitating activities that help reduce stress.
Table 3
Recommendations that the participating students identified when using the Objective Structured Clinical Evaluation (OSCE) in nursing
| Recommendations | Frequency | Percentile |
|---|---|---|
| Increasing time | 43 | 49.4 |
| Access to stations for practice or more previous preparation and practice | 15 | 17.2 |
| OSCE implementation | 14 | 16.0 |
| Carry out previous evaluations at the end of the course or tutoring before the process | 7 | 8.0 |
| Improve equipment and materials | 6 | 6.9 |
| Improvements to practice areas and stations | 6 | 6.9 |
| Stress reliefs | 3 | 3.4 |
| Better explanation of the OSCE and instructions | 6 | 3.4 |
| Human resources (ej. Greater support from the faculty) | 3 | 3.4 |
| Oportune citation and notification | 3 | 3.4 |
| Used for improvement | 2 | 2.3 |
| Evaluation with partner | 3 | 3.4 |
| Evaluation led by practice | 1 | 1.1 |
| Greater coordination | 1 | 1.1 |
| Improving the rubrics | 1 | 1.1 |
| Strengthen the physical test | 1 | 1.1 |
| Silence in the classroom | 1 | 1.1 |
The outcomes of the percentage and absolute distribution of students by questionnaire items and the identification of strengths revealed that participants perceived the OSCE as an aid tool in the evaluation of clinical and non-clinical nursing skills. The average scores obtained on the 5-point scale provided in the questionnaire reflected that the students tended to agree with the items that stated positive aspects of the OSCE. Students consider the OSCE very useful for evaluating various aspects such as clinical skills, ethical behavior, theoretical knowledge and communication skills. They consider it a stressful event. These results are consistent with the literature available in this research (Harden, et al., 1975; Rentschler, et al., 2007; Khattab & Rawlings, 2008; Alarcón, 2013).
Among other things, they recognized the OSCE as a learning experience that measures established goals and continues to be an effective facilitator for learning to perform clinical nursing skills as identified by Ross, et al. (1988). Therefore, this research demonstrates that hypothesis number 1; students perceive the OSCE as a tool to assist in the evaluation of clinical and non-clinical nursing skills, is positive. The analysis of the score expected by nursing students suggests that the OSCE facilitates the identification of the level of professional competence achieved by students at the end of nursing courses.
The perception of students who thought that they would have a score lower than 70% in the OSCE was less. However, the students’ own expressions of the OSCE’s strength and weaknesses demonstrates their appreciation of this tool for measuring their professional competencies. They also expressed the areas that must be improved in order to confront the challenges that they may face in their profession and in the formal practice of nursing with greater preparation. It is evident from this research that hypothesis number 2; The OSCE facilitates identifying the level of professional competence achieved by students at the end of a nursing course, is positive.
Incorporating the OSCE for professional competencies evaluation is an innovative educational experience in Puerto Rico that can contribute to the development of nursing curricula based on competencies. The estimated level reached in the achievement of the competences through the OSCE eases self-evaluation and planning to increase skill levels by completing different nursing courses before graduation. The knowledge of the viability and aspects to be improved in the use of the OSCE helps to identify the characteristics for its effectiveness in achieving the learning outcomes in nursing courses. Therefore, the knowledge acquired through the results of this research can help in best practices when using the OSCE in simulation centers in Puerto Rico.
Implications
Exploring the students’ opinions participating in this study revealed important aspects to consider when incorporating the OSCE in nursing programs in Puerto Rico. The respondent students’ answers were positive and represented an opportunity for their improvement and the achievement of their goals. Although the OSCE requires time and effort for its incorporation into the curriculum, its scope for the professional competences achievement remains significant in Puerto Rico as well as internationally.
This study demonstrates its efficacy and validity to assess competencies as required by accrediting agencies of nursing programs. Students suggest it be used in a summative and (or) formative manner in all nursing courses. They identified aspects that facilitate the OSCE such as: privacy, greater use of standardized patients, 10-15 minutes allotted time per station, greater practice and previous preparation before the OSCE, among others. Since a significant number of respondents experienced it as stressful, considering these aspects and the prior exploration of students’ knowledge and practices can help reduce interferences in their performance when evaluated through the OSCE.
The design and integration to carry out the OSCE is developed based on the characteristics and curricular programming of each discipline and institution. The identified need for the required materials and equipment implies recognizing cost-effective strategies or the possibility of considering an additional budget to comply with them as has been essential in some nursing programs that use the OSCE internationally. The OSCE is an assessment methodology used internationally in the recruitment of nursing professionals. By participating in it, graduates may be more prepared to better perform during recruitment processes in different countries.
Abdulghani, H. M., Ponnamperuma, Z. A., & Amin Z. (2015). An essential guide of developing, implementing, and evaluating Objective Structured Clinical Examination (OSCE). New Jersey: World Scientific Publishing Co.
Alarcón, M. A. (2013). Incorporación del Examen Clínico Objetivo Estructurado (ECOE) en la Carrera de Enfermería.Revista de Educación en Ciencias de la Salud, 10(1): 18-22. San Sebastián: Chile.Recuperado de https://dialnet.unirioja.es/servlet/articulo?codigo=4750349
Barry M., Noonan M., Bradshaw C., & Murphy-Tighe S. (2012). An exploration of student midwives’ experiences of the Objective Structured Clinical Examination assessment process.Nurse Education Today, 32(6), 690-4. doi:10.1016/j.nedt.2011.09.007
Bourke, M., & Ihrke, B. (2005). The evaluation process. In D. Billings & J. Halstead (Eds).Teaching in nursing: A guide for faculty(Pp.443-464). Philadelphia, PA: W. B. Saunders.
Caballero C., Creed F., Gochmanski C., & Lovegrove, J. (2012). Nursing OSCEs:a complete guide to exam success.New York: Oxford University Press.
Goñi, J. M. (2005).Competencias, tareas, y evaluación, los ejes del currículo universitario.Barcelona: Octaedro.
Harden R. M. & Gleeson F. A. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE).Medical Education, 13(1), 41- 54.
Harden R. M., Stevenson M., Downie W.W., & Wilson G. M. (1975). Assessment of clinical competence using objective structured examination.British Medical Journal, 1(5955), 447-451.
Harden, R. M. (2016). Revisiting 'Assessment of clinical competence using an objective structured clinical examination (OSCE)'.Medical Education, 50(4), 376-379. doi:10.1111/medu.12801
Hengstherger-Sims, C., Cowin, L. S., Eagar, S. C., Gregory. L. Andrew, S., & Rolley, J. (2008).elating new graduate nurse competence to frequency of use Collegian, 15(2), 69-76. doi:10.1016/j.colegn.2008.02.003
Jeffries, P. R., & Rogers, K. (2012). Theoretical framework for simulation design. In Jeffries, P. R. (Ed.),imulation in nursing education: from conceptualization to Evaluation(pp. 25-43). New York, NY: National League for Nursing.
Kelly, M., Mitchell, M., Jeffrey, C., Henderson, A., Groves, M., Nulty, D., Glover, P. & Knight, S. (2016). OSCE best practice guidelines, applicability for nursing simulations.Advances in Simulation, 1(10), 1-10 . doi 10.1186/s441077-016-0014-1
Khattab, A., & Rawlings, B. (2008). Use of a modified OSCE to assess nurse practitioner students.British Journal of Nursing, 17(12), 754-759.
Krippendorff, K. (2013).Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage Publications, Inc.
Meechan, R., Jones, H., & Valler-Jones, T. (2011). Do medicines OSCEs improve drug administration ability?British Journal of Nursing, 20(13), 817-822. Recuperado de http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?vid=9&sid=5e750c61-7ded-4b55-bb8f-3d3a35eff90a%40sessionmgr101
Merrifield, N. (2016).Trusts committed to nurse recruitment despite regulator headcount cuts plan.Recuperado de http://www.nursingtimes.net/news/workforce/exclusive-challenged-trusts-commit-to-recruiting-nurses/7002460.fullarticle
Moreno Corral, L. J. (2013).Mesa Redonda: La experiencia del Practicum. VII Jornadas de Profesorado de Facultades de Enfermería. Universidad de Cádiz. Recuperado de http://www.cnde.es/cms_files/VII_jornadas_La_experienci_practicum_Luis_Moreno.pdf
Oranye, N. O., Ahmad, Ch., Ahmad, N., & Abu Bakar, R. (2014). Assessing nursing clinical skills competence through objective structured clinical examination (OSCE) for open distance learning students in Open University Malaysia.Contemporary Nurse, 41(2), 233-241. Recuperado de http://dx.doi.org/10.5172/conu.2012.41.2.233
Rentschler, D. D., Eaton, J., Cappiello, J., McNally, SF., & McWilliam, P. (2007). Evalation of Undergraduate Students Using Objective Structured Clinical Evaluation. Journal of Nursing Education, 46(3), 135-139. Recuperado de http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?vid=34&sid=66a383a2-1d76-4717-8b94-42cbb5a36153%40sessionmgr101
Ross, M. C., G., Knight, J., Chamberlain, M., Fothergill-Bourbonnais, F., & Linton, J.(1988). Using the OSCE to measure clinical skills performance in nursingJournal of Advanced Nursing, 13(1) 45-56.
Scalabrini, A. (2015, 12 de marzo).Carta de respuesta del Dr. Scalabrini.Recuperado de https://outlook.office.com/owa/?realm=suagm.edu&exsvurl=1&ll-cc=20490&modurl=0&path=/mail/search
Stoll, M., Ard, N., & Vest, G. (2015).ACEN Accreditation Workshop.San Juan, Puerto Rico: ACEN.