Oral examinations have become more standardized over recent years. Traditionally a small number of raters were used for this type of examination. Past studies suggested that more raters should improve reliability. We compared the results of a multi-station structured oral examination using two different rater models, those based in a station, (station-specific raters), and those who follow a candidate throughout the entire examination, (candidate-specific raters).Two station-specific and two candidate-specific raters simultaneously evaluated internal medicine residents’ performance at each station. No significant differences were found in examination scores. Reliability was higher for the candidate-specific raters. Inter-rater reliability, internal consistency and a study of station inter-correlations suggested that a halo effect may be present for candidates examined by candidate-specific raters. This study suggests that although the model of candidate-specific raters was more reliable than the model of station-specific raters for the overall examination, the presence of a halo effect may influence individual examination outcomes.

During in-training assessment students are frequently assessed over a longer period of time and therefore it can be expected that their performance will improve. We studied whether there really is a measurable performance improvement when students are assessed over an extended period of time and how this improvement affects the reliability of the overall judgement. In-training assessment results were obtained from 104 students on rotation at our university hospital or at one of the six affiliated hospitals. Generalisability theory was used in combination with multilevel analysis to obtain reliability coefficients and to estimate the number of assessments needed for reliable overall judgement, both including and excluding performance improvement. Students’ clinical performance ratings improved significantly from a mean of 7.6 at the start to a mean of 7.8 at the end of their clerkship. When taking performance improvement into account, reliability coefficients were higher. The number of assessments needed to achieve a reliability of 0.80 or higher decreased from 17 to 11. Therefore, when studying reliability of in-training assessment, performance improvement should be considered.

Context: Various research studies haveexamined the question of whether expert ornon-expert raters, faculty or students,evaluators or standardized patients, give morereliable and valid summative assessments ofperformance on Objective Structured ClinicalExaminations (OSCEs). Less studied has beenthe question of whether or not non-facultyraters can provide formative feedback thatallows students to take advantage of theeducational opportunity that OSCEs provide. This question is becoming increasinglyimportant, however, as the strain on facultyresources increases.Methods: A questionnaire was developed toassess the quality of feedback that medicalexaminers provide during OSCEs. It was pilottested for reliability using video recordingsof OSCE performances. The questionnaires werethen used to evaluate the feedback given duringan actual OSCE in which clinical clerks,residents, and faculty were used as examinerson two randomly selected test stations.Results: The inter-rater reliabilityof the 19-item feedback questionnaire was 0.69during the pilot test. The internalconsistency was found to be 0.90 during pilottesting and 0.95 in the real OSCE. Using thisform, the feedback ratings assigned to clinicalclerks were significantly greater than thoseassigned to faculty evaluators. Furthermore,performance on the same OSCE stations eightmonths later was not impaired by having beenevaluated by student examiners.Discussion: While evidence of markinflation within the clinical clerk examinersshould be addressed with examiner training, thecurrent results suggest that clerks are capableof giving adequate formative feedback to morejunior colleagues.

Objective: A study aimed at exploring the variation in perceptions of learning outcomes reported by undergraduate nursing students enrolled in a problem-based learning subject in a pre-registration Bachelor of Nursing course (BN).

Method: Students were asked to respond to four open-ended questions which focussed on their learning outcomes in the different teaching/learning modalities of the subject. Data were analysed in two phases using a modified phenomenographic analysis. In the first phase a set of categories of description were developed from the student responses to questions related to the learning modalities. In the second phase the individual responses were classified in terms of the categories. Finally, correlations between the learning modalities were identified. In this paper the approach to analysis, the process of category identification and the correlations between the learning modalities will be described and the implications for further research and teaching will be discussed.

Results: The findings indicated that there were two distinct groups of student responses. Inward focussed students who described outcomes in terms of their own learning and students whose focus was outward i.e. describing learning in terms of patient care and how learning relates to that care. Another important result shows the relationship between the learning modalities and outcomes. From the students' perspective, the most sophisticated outcomes of the lectures and laboratories were ideas and skills to be used and applied in clinical settings. Whereas, the group-based activities in which clinical problems were presented to the students in the form of Situation Improvement Packages (SIPS) focussed their attention on the clinical setting which constituted a preparation for the realities of clinical practice.

Conclusion: The findings from this study indicate that students perceive their learning in the group based teaching/learning modality (SIPS) as effective in focussing them on the reality of their role in the clinical practice environment while lectures and laboratories provided the skills and knowledge required for this setting.

Directed self-guidance, whereby trainees independently practice a skill-set in a structured setting, may be an effective technique for novice training. Currently, however, most evaluation methods require an expert to be present during practice. The study aim was to determine if absolute symmetry error, a clinically important measure that can be assessed by the trainee, is a feasible assessment tool for self-guided learning of suturing skill. Forty-eight undergraduate medical trainees independently practiced suturing and knot tying skills using a benchtop model. Performance on a pretest, posttest, retention test and a transfer test was assessed using (1) the validated final product analysis (FPA), (2) the surgical efficiency score (SES), a combination of the FPA and hand motion analysis and (3) absolute symmetry error, a new measure that assesses the symmetry of the final product. Absolute symmetry error, along with the other objective assessment tools, detected improvements in performance from pretest to posttest (P < 0.05). A battery of correlation analyses indicated that absolute symmetry error correlates moderately with the FPA and SES. The development of valid, reliable and feasible technical skill assessments is needed to ensure all training centers evaluate trainee performance in a standardized fashion. Measures that do not require the use of experts or computers have potential for widespread use. We suggest that absolute symmetry error is a useful approximation of novices’ suturing and knot tying performance. Future research should evaluate whether absolute symmetry error can enhance learning when used as a source of feedback during self-guided practice.

Background: Change is ubiquitous. Current trends in both educational and clinical settings bring new challenges to clinicians and have the potential to threaten the quality of clinical teaching. Objective: To investigate hospital specialists’ perceptions of clinical teaching in the context of change. Design: Qualitative study using in-depth semi-structured interviews. Setting:Three hospital trusts in the United Kingdom associated with a new medical school Participants: A purposive sample of 15 clinicians from each of the three participating hospitals was approached. A total of 13 participated in the in-depth interviews: three from hospital A, six from hospital B and four from hospital C. Results: The two main themes of ‘Characteristics of Good and Bad Teachers’ and ‘Clinical Teaching Approaches’ emerged. These were underpinned by a number of sub-themes; including some seen by participants as potential barriers to teaching and learning, for example, organisational and personal issues. Conclusions: Potential barriers to teaching and learning, including why good clinical teachers may at times seen to be intimidating or to cause humiliation and problems of engaging with new educational practices, can be understood within the context of change resistance. Knowing more about how clinical teachers think about their task as educators is essential and this should underpin staff development and training programmes.

Background

Medical schools use supplemental peer-teaching programs even though there is little research on students’ actual experiences with this form of instruction.

Purpose

To understand the student experience of being taught by peers instead of by faculty.

Methods

We conducted focus groups with first- and second-year medical students participating in a supplemental peer-teaching program at one institution. From the learner focus group themes, we developed a questionnaire and surveyed all first-year students.

Results

Focus groups revealed four learner themes: learning from near-peers, exposure to second-year students, need for review and synthesis, teaching modalities and for the peer-teachers, the theme of benefits for the teacher. Factor analysis of the survey responses resulted in three factors: second-year students as teachers, the benefit of peer-teachers instead of faculty, and the peer-teaching process. Scores on these factors correlated with attendance in the peer-teaching program (P < .05).

Conclusions

Students valued learning from near-peers because of their recent experience with the materials and their ability to understand the students’ struggles in medical school. Students with the highest participation in the program valued the unique aspects of this kind of teaching most. Areas for improvement for this program were identified.

Physician competencies have increasingly been a focus of medical education at all levels. Although competencies are not a new concept, when the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) jointly agreed on six competencies for certification and maintenance of certification of physicians in 1999, it brought about renewed interest. This article gives a brief overview of how a competency-based curriculum differs from other approaches and then describes the issues that need to be considered in the design and implementation of such a curriculum. In order to achieve success, a competency-based curriculum requires careful planning, preparation and a long-term commitment from everyone involved in the educational process. Building a competency-based curriculum is really about maintaining quality control and relinquishing control to those who care the most about medical education, our students. In the face of the many challenges that are facing undergraduate medical education (UME), including declining availability of teaching patients and over-burdened faculty, instituting quality control and relinquishing control will be necessary to maintain high quality.