New frontiers in automated assessment: using latent semantic analysis to assess conceptual content in essays

Abstract

This article describes the use of latent semantic analysis (LSA), a machine-learning technique which has been developed for the computerised assessment of knowledge. LSA employs linear algebra techniques to induce and represent knowledge in high dimensional spaces, and can be used to compare documents in terms of their degree of semantic similarity to one another. In this article we report on a study to explore the feasibility of LSA as a computational tool for assessing the conceptual content of essay-type answers. Student answers to two short essay questions in an undergraduate psychology course were first independently graded by two human lecturers, and were then converted to machine readable texts and scored by LSA. The scores assigned by LSA showed good agreement with those awarded by the humans. The implications of the results, and some of the pros and cons of LSA as a practical assessment technology, are discussed.

South African Journal of Higher Education Vol. 19(6) 2005: 1074-1088