In search of the perceived quality and impact of accredited South African law journals: exploring the possibility of a ranking system. A baseline study: 2009 – 2014

Abstract

The DHET Research Output Policy (2015) indicates that there has been a change in the government’s approach to research funding. Previously all research published in any accredited journal was rewarded equally. A decision has been taken, however, that a shift will be made towards rewarding better quality and higher impact peer-review research. Additional mechanisms such as biometric/bibliometric data, including citations, assessments by discipline-specific panels of experts and/or post-publication reviews may be used to determine the quality and impact of publications. The policy notes that the DHET may distinguish between "high" and "low" impact journals after proper consultation.

This article highlights the need for consultation by the legal fraternity with the DHET about the implementation of these possible mechanisms in the light of the special considerations applicable to the evaluation of law journals: most journals publish mainly local legal content, there is a limited number of active legal academics, the nature of legal research is not empirical, and a premium is placed on the writing of books.

The research evaluates the available data between 2009 and 2014 in an attempt to assess if it would be appropriate to introduce a legal journal ranking system in South Africa. The article discusses direct and indirect forms of quality evaluation to inform possible ranking systems. This includes the data from the ASSAf expert panel evaluation of law journals in 2014 and other bibliometric data based on whether the journal is featured in international accredited lists, the size of its print-run, author prominence, rejection-rate, usage studies, and evaluations based on citations. An additional ranking system is considered, based on the five best outputs submitted to the National Research Foundation by applicants applying for rating.

The article concludes that a law journal ranking system would be inappropriate for South Africa. None of the systems meet the minimum requirements for a trustworthy ranking of South African law journals, as the data available are insufficient, non-verifiable and not based on objective quality-sensitive criteria. Consultation with the DHET is essential and urgent to avoid the implementation of inappropriate measures of quality and impact assessment.

Keywords: Ranking; law journals; Department of Higher Education; bibliometric data; citations; quality