Academic literacy tests: design, development, piloting and refinement

Abstract

Though there are many conditions for drafting language tests responsibly, this contribution focuses first on how to operationalise a set of three critically important design principles for such tests. For the last thirty years or so, developers of language tests have agreed that the most important design principle emanates from our ability to give a theoretical justification for what it is that we are measuring. Without this, we eventually have very little ground for a responsible interpretation of test results, which is a second, though not secondary, principle for language test design. There is a third principle involved, which is that the measuring instrument must be consistent and stable. The paper investigates how a blueprint for an academic literacy test may be conceptualised, how that could be operationalised, and demonstrates how pilot tests are analysed with a view to refining them. Finally, that leads to a consideration of how to arrive at a final draft test, and how valid and appropriate interpretations of its results may be made. Since the three conditions for language tests focussed on here are not the only design principles for such applied linguistic instruments, the discussion is placed in a broader philosophical framework for designing language tests that also includes a consideration of some of the remaining design principles for language testing.