Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results

Abstract

Weighting of items in an evaluation instrument contributes to more meaningful and valid interpretations of student performance in respect of each learning outcome or item being assessed. It follows that the validity of instruments is important for meaningful inferences about students’
learning performance, including their performance in tutorial groups. The Delphi technique was used to elicit experts’ subjective judgement of the content validity of items in the tutorial performance evaluation instrument in rounds one and two. A sample of eight experts (n = 8) was selected by purposive, maximum variation sampling. In round three Delphi a weighted score was determined for each of the instrument items, subitems
and Likert scale points through pairwise comparison by the experts. Mathematical modelling of experts’ weighting comparisons, recorded on visual analogue scales, resulted in proportional weights for each item; these weights are expressed as a percentage. The fi nal instrument comprised weighted items measured on a rating scale with points that are
not equidistant. A computerised tutorial performance evaluator (TPE) was developed for accurate, economical and effi cient calculation of student scores. The purpose of this article is to report on the statistical analysis and results of the weighting of items in an instrument to assess and evaluate baccalaureate nursing students’ performance in problem-based learning tutorials.

Die waardebepaling van items in ‘n evalueringsinstrument dra by tot ‘n meer betekenisvolle en geldige interpretasie van studente se vermoë ten opsigte van elke leeruitkoms of item wat geassesseer word. Hieruit volg dit dat die geldigheid van instrumente belangrik is vir betekenisvolleafl eidings betreffende studente se leervermoë, insluitend hulle prestasie in leergroepe. Die Delphi-tegniek is gebruik in rondtes een en twee om kundiges se subjektiewe oordeel oor die inhoudsgeldigheid van items in die leergroepgedragsevalueringsinstrument te bekom. ‘n Steekproef van agt kundiges (n = 8) is deur doelgerigte, maksimale-variasiesteekproeftrekking gekies. Die kundiges het in rondte drie van die Delphi-tegniektoepassing deur middel van gepaarde vergelyking ‘n waarde bepaal vir elk van die instrumentitems, sub-items en Likertskaalpunte. Die
wiskundige modellering van kundiges se waardebepalingsvergelykings, aangeteken op visuele analogiese skale, het proporsionele gewigte vir elke item tot gevolg gehad. Hierdie gewigte word persentasiegewys voorgestel. Die fi nale instrument het bestaan uit items wat gemeet word teen ‘n graderingskaal met punte wat nie op ‘n gelyke afstand van mekaar is nie. ‘n Rekenaargebaseerde leergroepgedragsevalueringsinstrument is ontwikkel vir die akkurate, ekonomiese en doeltreffende berekening van studente se punte. Die doel van die artikel is om verslag te lewer oor die statistiese analise en resultate van die waardebepaling van items in ‘n instrument om baccalaureate-verpleegkundestudente se prestasie in probleemgebaseerde leergroepe te meet en te evalueer.