Levelling the playing field: an investigation into the translation of academic literacy tests

Abstract

It is widely accepted that low levels of proficiency in the languages of learning and teaching (in this article, academic language proficiency refers to academic literacy and the terms will be used interchangeably), affect through-put rates negatively. This unsettling trend is confirmed by local and international literature, and can possibly be attributed to the language curriculum in secondary education that does not prepare students adequately for the higher-order language-thinking skills they need for study at university. By this we refer to Bloom’s taxonomy, especially the three higher-order skills of analysing, synthesising and evaluating, and the way language is used for these purposes. In order to address this problem, and as part of language-planning initiatives, some faculties at Stellenbosch University introduced the integration of academic literacy courses into the first-year curriculum. These courses are fully creditbearing and a system of continuous assessment was adopted. Semester tests form part of this assessment process, and led to the investigation done for this paper. Since both Afrikaans- and English-speaking students register for the same academic literacy module it is imperative that outcomes and assessments should be on the same level. However, the aggregate on the Afrikaans semester tests have continuously been lower than on the English test. The aggregate for the Afrikaans tests was, furthermore, on par with the weighted average for all other first-year courses, which was not always the case with the English tests. After an initial investigation, it was concluded that the Englishspeaking students were not necessarily academically stronger than their Afrikaans counterparts, but it seemed likely that the problem lay with the tests themselves. A first notion was that academic and spoken English are closer than academic and spoken Afrikaans. It was also possible that the level of difficulty of the English test was substantially lower than that of the Afrikaans test. It should, however, be noted that both the Afrikaans and English tests produced excellent reliability coefficients (alpha above 0.88) and most items discriminated adequately. A possible solution to the benchmarking problem was to translate the Afrikaans test into English. The translation framework, adopted for this study, was Nord’s functionalist model. This paper will elaborate on the translation procedure, and the variance in students’ performance on the translated version compared to previous administrations. Preliminary conclusions on bias in translated tests and the success and feasibility of such procedures are drawn.

Daar word allerweë aanvaar dat lae vaardigheidsvlakke in die tale van onderrig en leer (in hierdie artikel verwys akademiese taalvaardigheid na akademiese geletterdheid en die twee terme word afwisselend met mekaar gebruik) deurvloeikoerse benadeel. Hierdie onrusbarende tendens word deur plaaslike én internasionale literatuur bevestig, en kan moontlik daaraan toegeskryf word dat die taalkurrikulum in hoërskoolonderrig nie daarin slaag om studente genoegsaam vir die hoërorde-taaldenkvaardighede van universiteitstudie toe te rus nie. Met hoërorde-taaldenkvaardighede word verwys na Bloom se taksonomie, en spesifiek die drie hoërorde-vaardighede van analise, sintese en evaluering, en die wyse waarop taal vir hierdie doel aangewend word. Om hierdie probleem te hanteer, en as deel van taalbeplanningsinisiatiewe, het sommige fakulteite aan die Universiteit Stellenbosch kursusse in Akademiese Geletterdheid by die eerstejaarskurrikulum begin integreer. Hierdie kursusse is ten volle kredietdraend en volg ‘n stelsel van voortgesette assessering. Semestertoetse maak deel uit van hierdie assesseringsproses en het tot die ondersoek vir hierdie navorsingstuk aanleiding gegee. Aangesien sowel Afrikaans- as Engelssprekende studente vir dieselfde module in Akademiese Geletterdheid registreer, is dit noodsaaklik dat uitkomste en assessering op gelyke vlak lê. Tog was die totale punt vir die Afrikaanse semestertoets deurgaans laer as dié vir die Engelse toets. Voorts was die totale punt vir die Afrikaanse toets in lyn met die geweegde gemiddelde vir alle ander eerstejaarskursusse, wat weer nie altyd die geval met die Engelse toets was nie. Ná ‘n aanvanklike ondersoek is afgelei dat die Engelssprekende studente nie noodwendig akademies sterker as hul Afrikaanse eweknieë is nie, maar dat die probleem waarskynlik by die toetse self lê. ‘n Eerste moontlikheid was dat akademiese en gesproke Engels nader aan mekaar is as akademiese en gesproke Afrikaans. Tweedens kon die moeilikheidsgraad van die Engelse toets ook aansienlik laer wees as dié van die Afrikaanse toets. Dit is egter belangrik om daarop te let dat die Afrikaanse én Engelse toetse albei uitstekende betroubaarheidskoëffisiënte (alfa bo 0.88) sowel as merendeels goeie diskrimineringskoëffisiënte opgelewer het. ‘n Moontlike oplossing vir die probleem van rigpuntstelling was om die Afrikaanse toets in Engels te vertaal. Die vertaalraamwerk wat vir hierdie studie aanvaar is, was Nord se funksionalistiese model. Hierdie navorsingstuk wei uit oor die vertaalprosedure sowel as die wisseling in studenteprestasie in die vertaalde toets in vergelyking met vorige toetse. Voorlopige gevolgtrekkings word ook gemaak oor sydigheid in vertaalde toetse, en die sukses en uitvoerbaarheid van sodanige prosedures.

Keywords: academic literacy test; functionalist translation approach; back-translation; adaptation of tests

The article is in English.