Article Sidebar
Published:
Sep 11, 2022Article Details
Copyright remains in the Author’s name. The work is licensed under a Creative Commons Attribution - Noncommercial Works License. Authors are required to complete and sign an Author Agreement form that outlines Author and Publisher rights and terms of publication. The Agreement form should be uploaded along with other submissions files and any submission will be considered incomplete without it [forthcoming].
Material submitted for publication in the AJHPE is accepted provided it has not been published or submitted for publication elsewhere. Please inform the editorial team if the main findings of your paper have been presented at a conference and published in abstract form, to avoid copyright infringement. The AJHPE does not hold itself responsible for statements made by the authors. The corresponding author should also indicate if the research forms part of a postgraduate short report, dissertation or thesis.
Previously published images
If an image/figure has been previously published, permission to reproduce or alter it must be obtained by the authors from the original publisher and the figure legend must give full credit to the original source. This credit should be accompanied by a letter indicating that permission to reproduce the image has been granted to the author/s. This letter should be uploaded as a supplementary file during submission.Main Article Content
Evaluation of assessment marks in the clinical years of an undergraduate medical training programme: Where are we and how can we improve?
Abstract
Background. In high-stakes assessments, the accuracy and consistency of the decision to pass or fail a student is as important as the reliability of the assessment.
Objective. To evaluate the reliability of results of high-stakes assessments in the clinical phase of the undergraduate medical programme at the University of the Free State, as a step to make recommendations for improving quality assessment.
Methods. A cohort analytical study design was used. The final, end-of-block marks and the end-of-year assessment marks of both fourth-year and final-year medical students over 3 years were compared for decision reliability, test-retest reliability, stability and reproducibility.
Results. 1 380 marks in 26 assessments were evaluated. The G-index of agreement for decision reliability ranged from 0.86 to 0.98. In 88.9% of assessments, the test-retest correlation coefficient was <0.7. Mean marks for end-of-block and end-of-year assessments were similar. However, the standard deviations of differences between end-of-block and end-of-year assessment marks were high. Multiple-choice questions (MCQs) and objective structured clinical examinations (OSCEs) yielded good reliability results.
Conclusion. The reliability of pass/fail outcome decisions was good. The test reliability, as well as stability and reproducibility of individual student marks, could not be accurately replicated. The use of MCQs and OSCEs are practical examples of where the number of assessments can be increased to improve reliability. In order to increase the number of assessments and to reduce the stress of high-stake assessments, more workplace-based assessment with observed clinical cases is recommended.
