Comparing university rankings

Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.     

A comparative study on world university rankings: a bibliometric survey

Recently there are many organizations conducting projects on ranking world universities from different perspectives. These ranking activities have made impacts and caused controversy. This study does not favor using bibliometric indicators to evaluate universities?? performances, but not against the idea either. We regard these ranking activities as important phenomena and aim to investigate correlation of different ranking systems taking bibliometric approach. Four research questions are discussed: (1) the inter-correlation among different ranking systems; (2) the intra-correlation within ranking systems; (3) the correlation of indicators across ranking systems; and (4) the impact of different citation indexes on rankings. The preliminary results show that 55?% of top 200 universities are covered in all ranking systems. The rankings of ARWU and PRSPWU show stronger correlation. With inclusion of another ranking, WRWU (2009?C2010), these rankings tend to converge. In addition, intra-correlation is significant and this means that it is possible to find out some ranking indicators with high degree of discriminativeness or representativeness. Finally, it is found that there is no significant impact of using different citation indexes on the ranking results for top 200 universities.     

A critical comparative analysis of five world university rankings

Scientometrics - To provide users insight into the value and limits of world university rankings, a comparative analysis is conducted of five ranking systems: ARWU, Leiden, THE, QS and U-Multirank....     

Country-specific determinants of world university rankings

This paper examines country-specific factors that affect the three most influential world university rankings (the Academic Ranking of World Universities, the QS World University Ranking, and the Times Higher Education World University Ranking). We run a cross sectional regression that covers 42–71 countries (depending on the ranking and data availability). We show that the position of universities from a country in the ranking is determined by the following country-specific variables: economic potential of the country, research and development expenditure, long-term political stability (freedom from war, occupation, coups and major changes in the political system), and institutional variables, including government effectiveness.     

Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings

We applied a set of standard bibliometric indicators to monitor the scientific state-of-arte of 500 universities worldwide and constructed a ranking on the basis of these indicators (Leiden Ranking 2010). We find a dramatic and hitherto largely underestimated language effect in the bibliometric, citation-based measurements of research performance when comparing the ranking based on all Web of Science (WoS) covered publications and on only English WoS covered publications, particularly for Germany and France.     

Self-defined information indices: application to the case of university rankings

Scientometrics - University rankings are now relevant decision-making tools for both institutional and private purposes in the management of higher education and research. However, they are often...     

An insight into the importance of national university rankings in an international context: the case of the I-UGR rankings of Spanish universities

The great importance international rankings have achieved in the research policy arena warns against many threats consequence of the flaws and shortcomings these tools present. One of them has to do with the inability to accurately represent national university systems as their original purpose is only to rank world-class universities. Another one has to do with the lack of representativeness of universities’ disciplinary profiles as they usually provide a unique table. Although some rankings offer a great coverage and others offer league tables by fields, no international ranking does both. In order to surpass such limitation from a research policy viewpoint, this paper analyzes the possibility of using national rankings in order to complement international rankings. For this, we analyze the Spanish university system as a study case presenting the I-UGR Rankings for Spanish universities by fields and subfields. Then, we compare their results with those obtained by the Shanghai Ranking, the QS Ranking, the Leiden Ranking and the NTU Ranking, as they all have basic common grounds which allow such comparison. We conclude that it is advisable to use national rankings in order to complement international rankings, however we observe that this must be done with certain caution as they differ on the methodology employed as well as on the construction of the fields.     

Applying the CSS method to bibliometric indicators used in (university) rankings

Scientometrics - This Letter to the Editor proposes to use the CSS method for classifying ranking results (e.g. from university rankings) into meaningful groups.     

Improving quality assessment of composite indicators in university rankings: a case study of French and German universities of excellence

Composite indicators play an essential role for benchmarking higher education institutions. One of the main sources of uncertainty building composite indicators and, undoubtedly, the most debated problem in building composite indicators is the weighting schemes (assigning weights to the simple indicators or subindicators) together with the aggregation schemes (final composite indicator formula). Except the ideal situation where weights are provided by the theory, there clearly is a need for improving quality assessment of the final rank linked with a fixed vector of weights. We propose to use simulation techniques to generate random perturbations around any initial vector of weights to obtain robust and reliable ranks allowing to rank universities in a range bracket. The proposed methodology is general enough to be applied no matter the weighting scheme used for the composite indicator. The immediate benefit achieved is a reduction of the uncertainty associated with the assessment of a specific rank which is not representative of the real performance of the university, and an improvement of the quality assessment of composite indicators used to rank. To illustrate the proposed methodology we rank the French and the German universities involved in their respective 2008 Excellence Initiatives.     

Iberian universities: a characterisation from ESI rankings

The access to bibliographic and citation databases allows to evaluate scientific performance, and provides useful means of general characterisation. In this paper we investigate the clustering of Iberian universities, resulting from the similarity in the number and specific nature of the scientific disciplines given by the Essential Science Indicators database. A further refining of the analysis, as provided by PCA, clearly reveals the relationship between the universities and the scientific disciplines in the main groups. Similarity between universities is not dictated only by the number of areas in the ranking, but also stems from the nature of the ranked scientific areas and the specific combination in each university.     

Understanding images: an approach to the university teaching ofcomputer vision

Computer vision is a unit of several degree courses. The authors believe that, regardless of the educational context, a common approach to computer vision teaching can be adopted by centring on real-world uses of the technology. Only through detailed consideration of the relationship between the world, the images it generates and the visual task to be performed can students learn the basic skill required for the development of successful computer vision systems: how to evaluate and identify appropriate prior knowledge. This article gives some examples of this approach to teaching computer vision and of the resources required     

A mathematical model to optimize decisions to impact multi-attribute rankings

We formulate the problem of how to climb in multi-attribute rankings with known weights using mathematical optimization. A model is derived based on familiar practices used in rankings in higher education where several attributes are combined using known weights to obtain a score. The method applies in any situation where multiple attributes are used to rank entities. We invoke several assumptions such as independence among attributes and that administrators can affect the values of some of the attributes and know the cost of doing so. Our results suggest that a strategy to advance in the rankings is to focus on modifying the value of fewer rather than more attributes. The model is generalized to allow for synergies and antagonisms among the attributes.     

The model of maximum productivity for research universities SciVal author ranks,productivity, university rankings,and their implications

Scientometrics - Aware of the growing importance of global rankings, universities aim to enhance their positions. However, the exact relation between research productivity and ranking positions is...     

Research performance indicators for university departments: A study of an agricultural university

The present bibliometric study extends previous work by focusing on the research performance of departments in the natural and life sciences, the social and behavioral sciences, and the humanities. The present study covers all 70 departments from one agricultural university, and several veterinary departments of a second university. The impact analysis was extended by including other types of documents than journal articles. For about a third of the departments, publications not covered in citation indexes accounted for at least 30% of the citations to their total oeuvre. To deal with different citation and publication habits in the various fields, both short-term and medium-term impact assessments were made. The commonly used three year window is not universally applicable, as our results show. The inclusion of self-citations forms an important source of error in the ratio of actual/expected impact. To cope with this, the trend and level of self-citations was compared at university level with that in a matched sample of publications. Moreover, at a departmental level, self-citation rates were used to detect departments with divergent levels of self-citation. The expected impact of journals accounted for only 18% of the variance in actual impact. Comparison of bibliometric indicators with two peer evaluations showed that the bibliometric impact analyses provided important additional information.     

Analyzing the impact of reputational bias on global university rankings based on objective research performance data: the case of the Shanghai Ranking (ARWU)

Scientometrics - This paper analyzes the potential impact of reputational bias on academic classifications based upon current research performance measures. Empirical evidence supports the...     

Institutional research rankings via bibliometric analysis and direct peer review: A comparative case study with policy implications

Recent years have seen enormously increased interest in the comparative evaluation of research quality in the UK, with considerable resources devoted to ranking the output of academic institutions relative to one another at the sub-discipline level, and the disposition of even greater resources dependent on the outcome of this process. The preferred methodology has been that of traditional peer review, with expert groups of academics tasked to assess the relative worth of all research activity in ‘their’ field. Extension toinstitutional evaluation of a recently refined technique ofjournal ranking (Discipline Contribution Scoring) holds out the possibility of ‘automatic’ evaluation within a time-frame considerably less than would be required using methods based directly on citation counts within the corpus of academic work under review. This paper tests the feasibility of the technique in the sub-field of Business and Management Studies Research, producing rankings which are highly correlated with those generated by the much more complex and expensive direct peer review approach. More generally, the analysis also gives a rare opportunity directly to compare the equivalence of peer review bibliometric analysis over a whole sub-field of academic activity in a non-experimental setting.     

Metrology-cybernetics-control: A new approach

As a science and as a special activity concerned with measurements, the science of metrology pursues a classical, orthodox purpose, mainly involving the measurement of physical quantities. The term “measurement” is now applied also to social, economic, medical, and other disciplines, including cybernetics and problems of management and control. Translated from Izmeritel'naya Tekhnika, No. 7, pp. 17–20, July, 1996.     

Uncovering excellence in academic rankings: a closer look at the Shanghai ranking

In this paper, we examine whether the quality of academic research can be accurately captured by a single aggregated measure such as a ranking. With Shanghai University’s Academic Ranking of World Universities as the basis for our study, we use robust principal component analysis to uncover the underlying factors measured by this ranking. Based on a sample containing the top 150 ranked universities, we find evidence that, for the majority of these institutions, the Shanghai rankings reflect not one but in fact two different and uncorrelated aspects of academic research: overall research output and top-notch researchers. Consequently, the relative weight placed upon these two factors determines to a large extent the final ranking.