Drifts and Pernicious Effects of the Quantitative Evaluation of Research: The Misuse of Bibliometrics

Abstract

The focus of this article is on the mechanisms behind the bibliometric indicators (principally journal impact factors and researchers’ h-indexes) used to quantify the creative productivity and academic influence of scientists and scientific periodicals. The increasing use of these indicators in the academic world is associated with the increasing dominance of the managerial approach to higher education and science based on the neoliberal logic of efficiency and cost optimization. A critical analysis of the algorithms that calculate these indicators demonstrates a number of asymmetries and false logical assumptions that have a negative impact on the dynamics of scientific research, especially in the field of social sciences and humanities. For example, the well-known ratings lag between journals in the hard sciences, social sciences and humanities results from the way “impact factors” (the arithmetic mean of the number of mentions of articles of a particular journal over a two-year period) are calculated without considering the differing tempos at which these disciplines evolve. Considering the “h-index” and the basis of its calculation, a no less fundamental flaw of axiology can be found in the very preconception that a quantitative indicator speaks to the real value of a scientist’s work for their particular field of research. Moreover, the possibility of manipulation, sewn into the very nature of bibliometric indicators, has generated a significant number of tricks and schemes that allow for specific researchers and institutions to increase their ratings. It has likewise been used by university and academic administrations as a tool to formally legalize often discriminatory decisions in the allocation of resources.