Towards transparency and accountability: The story of the Test of Academic Literacy for Postgraduate Students (TALPS)

Abstract

Unfair tests, unfair testing methods and the use of tests to restrict and deny access have ensured a negative attitude to tests. The move in the recent past (Shohamy 2001, 2008; Fulcher & Davidson 2007; McNamara & Roever 2006) has been to promote the design and development of fair tests, by test developers who are willing to be accountable for their designs. A first step in this process is to ensure that every step of the design process is documented, and that this information is available to the public. Making this information available means that test takers are now equipped with information about the test and can now ask questions about the test. Test developers become real, not just experts ‘hiding behind their designs’. Importantly, this kind of transparency ensures a channel of communication, not just between test developers and other experts in the field, but also between test developers and test takers. Applied linguists should strive to ensure that the tests they design and use are fair, socially acceptable and have positive effects. This paper will illustrate that these concerns become important when one works within a framework that challenges test developers to consider questions related to every aspect of
the test. In employing a framework that incorporates a concern for the empirical analyses of a test, as well as a concern for the social dimensions of language testing, one is compelled to ensure transparency and  accountability in the testing process, as well as giving a voice to those often ignored, but most affected by the use of the test scores: the test-takers. In telling the story of the design and development of TALPS, this article is the first step in ensuring the transparency and accountability of the test developers of TALPS.

Keywords: academic literacy, construct, specifications, task types, sub-tests, descriptive statistics, transparency, accountability