Predictability of the must (math-up skills test)

Abstract

In the USA for the most part, completion of a first-semester general chemistry (Chem I) course lays the foundation deemed necessary for  understanding second-semester general chemistry (Chem II) topics. Successful completion of Chem I and II gives students permission to  progress to organic chemistry I (O-Chem). A series of studies undertaken by the NSA (Networking for Science Advancement) Texas team  began in 2016. Texas is one of five majority-minority states in the USA and hosts a significant Hispanic population. The purpose of this  research line is to evaluate the influence of basic arithmetic automaticity (what students can do without a calculator) skills needed to  succeed in lower-level chemistry. Over 9,000 students from nine universities have contributed to this research. Results suggest a strong  correlation between procedural arithmetic preparation, automaticity, and student performance in Chem I, II, and O-Chem courses. The  NSA collaborative uses the Math-Up Skills Test (MUST) as an assessment instrument along with student demographics to identify at-risk  students from these contributing populations at the beginning of a course with high reliability (KR-20 = 0.863) and effect size (Cohen's d ≥ 1.20). The hand-graded MUST requires only 15 minutes of class time to administer and combined with specific demographic categories consistently predicts students’ success rate in lower-level chemistry about 80 percent of the time therefore providing adequate time to  identify and help at-risk students. This paper is about the evolution of the MUST and how following the NSA team's research line has  advanced its use and interpretation.