A Comparative Analysis of the Procedure Employed in Item Generation and Establishing Reliability in Selected Nigerian and Non-Nigerian Educational and Psychological Tests

Abstract

The study examined the procedures for generating individual items on selected Nigerian and other non-Nigerian educational and psychological tests. Furthermore, it investigated the process of establishing their reliability. The population for the study consisted of validated Nigerian and non-Nigerian educational and psychological tests. Thirty validated educational and psychological scales designed to measure constructs in education and social sciences were purposively selected for the study based on accessibility and availability of validation information. The instruments used for the study were scaling procedures used in 27 published journal articles from issues of seven published journals. Data collected were analysed based on the criteria used in generating items and establishing reliabilities. The results showed that literature review was the highly used criterion for item generation (N=19, 48.7%) while theory and responses to open-ended questions were next (N=7, 17.95% and N=6, 15.38%) respectively. Internal consistency reliability was reported most frequently using Cronbach’s alpha (N=27, 56.26%), Guttman’s split-half reliability (N=4, 8.33%) and Spearman Brown’s split-half reliability (N=3, 6.26%). The test-retest method (N=10, 20.83%), Theta reliability (N=1, 2.08%), Omega reliability (N=1, 2.08%) and unspecified others (N=1, 2.08%) were other approaches used in determining the reliability of the scales studied. The study concluded that there was uniformity in the approaches employed in the development of educational and psychological scales.  While the most widely acceptable method for item generation was literature review; Cronbach’s alpha internal consistency was the widely acceptable method for reporting reliability.