The Dilemma of Standard Setting for the OSCE

Abstract

Background: Recently disparities between the OSCE raw scores and global scores have resulted in the need treatment of the raw scores in different ways such as borderline regression and borderline group regression. The object of this paper is to present station scoring forms designed to satisfy predetermined criteria and minimum pass levels.

Methods: Available samples of marking sheets and checklists designed by various
examining bodies were scrutinized. Criteria were prioritized according to commonly used grading systems. The station rating scale (check list) was designed to allow the observer to concentrate on checking the performance of the candidate without marking at the same time. The station marks form enables entry of marks based on the criteria in the Station rating scale.

Results: Three forms were designed. Forms 1 & 2 should be prepared beforehand with real or standardized patients. Form 3 is a combination to be used when last minute stations are introduced. The mark allocated to each observable criterion is made within the limits of the specified criteria. The global score has been retained to check for inconsistencies and for longitudinal studies on validity and reliability.

Conclusions: Prototype forms are presented; using predetermined, categorized grading criteria. The forms enable examiners to separate the observation stage from the actual allocation of marks. As in all OSCE settings, objectivity, validity and reliability will depend on prioritizing the selection of stations, clarity of the selected criteria and the training of examiners.