OSCEs for undergraduate clinical examination in orthopaedics: inter-examiner variability

Abstract

Background: The traditional clinical examination has fallen into disfavour on account of considerable inter-examiner variability. The OSCE is gaining popularity as it is perceived to be less prone to this.

Objective: To establish whether inter-examiner variability is still a significant factor for the undergraduate orthopaedic clinical examination in our institution.

Method: Thirty three final year students were randomly divided into two groups of 17 and 16 students. Two standardized OSCE questions were administered to each student by four examiners with each group being examined by one lecturer for each of the questions. For the first question, students in Group 1 were examined by Examiner A while those in Group 2 were examined by Examiner B. For the second question, students in Group 1 were examined by Examiner C while those in Group 2 were examined by Examiner D. The scores for each student were tabulated and the range, mean, and pass rate determined for each of the examiners. The Student’s t-test was calculated to determine if there was statistically significant interexaminer variability.

Results: For Question 1, the mean score for students examined by Examiner A (Group 1) was 7.47 marks while that for Examiner B (Group 2) was 5.59 marks. The p-value was 0.01367 (95% confidence interval). For Question 2, the mean score for students examined by Examiner C (Group 1) was 7.32 marks while that for Examiner D (Group 2) was 8.625. The p-value was 0.001148 (95% confidence interval).

Conclusion: There was statistically significant inter-examiner variability. We recommend that for all OSCE exams, examiners be paired with a deliberate attempt to pair a “Hawk” with a “Dove”. Statistical correction of biases is also recommended.