World-wide university rankings: A Scandinavian approach

Although universities’ world rankings are popular, their design and methods still request considerable elaborations. The paper demonstrates some shortcomings in the Academic World Ranking of Universities (ARWU, Shanghai Jiao Tong University) ranking methods. One deficiency is that universities’ scale differences are neglected due to omitting the whole input side. By resampling and reanalyzing the ARWU data, the paper proposes an input-output analysis for measuring universities’ scientific productivity with special emphasis on those universities which meet the productivity threshold (i.e. share of output exceeds share of input) in a certain group of universities. The productivity analysis on Scandinavian universities evaluates multidisciplinary and specialized universities on their own terms; consequently the ranking based on scientific productivity deviates significantly from the ARWU.     

How does the normalization of data affect the ARWU ranking?

The aim of this paper is to present new ideas in evaluating Shanghai University??s Academic Ranking of World Universities (ARWU). In particular, this paper shall try to determine whether the normalization of data affects University ranks. In accordance with this, both the normalized and original (raw) data for each of the six variables has been obtained. Based on a sample containing the 54 US universities which are placed in the ARWU top 100, the statistical I-distance method was performed. The results showed great inconsistencies between university ranks obtained for the original and normalized data. These findings were then analyzed and the universities that had the greatest fluctuation in their ranks were noted.     

On the internal dynamics of the Shanghai ranking

The Academic Ranking of World Universities (ARWU) published by researchers at Shanghai Jiao Tong University has become a major source of information for university administrators, country officials, students and the public at large. Recent discoveries regarding its internal dynamics allow the inversion of published ARWU indicator scores to reconstruct raw scores for 500 world class universities. This paper explores raw scores in the ARWU and in other contests to contrast the dynamics of rank-driven and score-driven tables, and to explain why the ARWU ranking is a score-driven procedure. We show that the ARWU indicators constitute sub-scales of a single factor accounting for research performance, and provide an account of the system of gains and non-linearities used by ARWU. The paper discusses the non-linearities selected by ARWU, concluding that they are designed to represent the regressive character of indicators measuring research performance. We propose that the utility and usability of the ARWU could be greatly improved by replacing the unwanted dynamical effects of the annual re-scaling based on raw scores of the best performers.     

Adjusted sum of institutional scores as an indicator of the presence of university systems in the ARWU ranking

In this article I introduce a new indicator that measures the presence of a higher education system in the Shanghai Jiao Tong Academic Ranking of World Universities (ARWU). First, the benefits of introducing such a measure and the drawbacks associated with the possible choices of the indicator are discussed. To analyze the drawbacks, the sample of countries with presence in ARWU is split into two groups of small and large world’s GDP share. A raw indicator based upon the sum of the scores of all the universities from a country divided by its world’s GDP share shows a noticeable bias in favor of small countries, so a one-way between-groups analysis of variance is conducted to help in canceling the bias. That leads to the introduction of a new aggregate indicator that can be computed in a very simple fashion. A discussion of the performance of higher education systems using this new indicator closes the paper.     

A methodology for Institution-Field ranking based on a bidimensional analysis: the <Emphasis Type="Italic">IFQ</Emphasis><Superscript>2</Superscript><Emphasis Type="Italic">A</Emphasis> index

The problem of comparing academic institutions in terms of their research production is nowadays a priority issue. This paper proposes a relative bidimensional index that takes into account both the net production and the quality of it, as an attempt to provide a comprehensive and objective way to compare the research output of different institutions in a specific field, using journal contributions and citations. The proposed index is then applied, as a case study, to rank the top Spanish universities in the fields of Chemistry and Computer Science in the period ranging from 2000 until 2009. A comparison with the top 50 universities in the ARWU rankings is also made, showing the proposed ranking is better suited to distinguish among non-elite universities.     

University patenting activities and their link to the quantity and quality of scientific publications

Integrating data from three independent data sources––USPTO patenting data, Shanghai Jiao Tong University’s Academic Ranking of World Universities (ARWU) and the Times Higher Education Supplement’s World University Ranking (WUR), we examine the possible link between patenting output and the quantity and quality of scientific publications among 281 leading universities world-wide. We found that patenting by these universities, as measured by patents granted by the USPTO, has grown consistently faster than overall US patenting over 1977–2000, although it has grown more slowly over the last 5 years (2000–2005). Moreover, since the mid-1990s, patenting growth has been faster among universities outside North America than among those within North America. We also found that the patenting output of the universities over 2003–2005 is significantly correlated with the quantity and quality of their scientific publications. However, significant regional variations are found: for universities in North America, both the quantity and quality of scientific publications matter, but for European and Australian/NZ universities, only the quantity of publications matter, while for other universities outside North America and Europe/Australia/NZ, only quality of publications matter. We found similar findings when using EPO patenting data instead of USPTO data. Additionally, for USPTO data only, the degree of internationalization of faculty members is found to reduce patenting performance among North American universities, but to increase that of universities outside North America. Plausible explanations for these empirical observations and implications for future research are discussed.     

On using the Shanghai ranking to assess the research performance of university systems

We take a new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems. We deal with system aggregates by means of averaging scores taken over a number of institutions from each higher education system according to the Gross Domestic Product of its country. We treat the set of indicators (measures) at the country level as a scale, and investigate its reliability and dimensionality using appropriate statistical tools. After a Principal Component Analysis is performed, a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale, with a first component that explains more than 72% of the variance of the sample under analysis. The percentages of variance of the indicators explained by the first component do shed light on the fact that ARWU is in fact measuring the research quality (both at the individual and collective levels) of a university system. When the second principal component is taken into account, the two principal components contribute to explain more than 90% of the variance. The rotated solution facilitates the interpretation of the components and provides clear and interesting clustering information about the 32 higher education systems under analysis.     

What do global university rankings really measure? The search for the X factor and the X entity

Most academic rankings attempt to measure the quality of university education and research. However, previous studies that examine the most influential rankings conclude that the variables they use could be an epiphenomenon of an X factor that has little to do with quality. The aim of this study is to investigate the existence of this hidden factor or profile in the two most influential global university rankings in the world: the Academic Ranking of World Universities (ARWU) of the University of Shanghai Jiao Tong, and the Times Higher Education (THE) ranking. Results support the existence of an underlying entity profile, characterized by institutions normally from the US that enjoy a high reputation. Results also support the idea that rankings lack the capacity to assess university quality in all its complexity, and two strategies are suggested in relation to the vicious circle created between institutional reputation and rankings.     

A comparative study on world university rankings: a bibliometric survey

Recently there are many organizations conducting projects on ranking world universities from different perspectives. These ranking activities have made impacts and caused controversy. This study does not favor using bibliometric indicators to evaluate universities?? performances, but not against the idea either. We regard these ranking activities as important phenomena and aim to investigate correlation of different ranking systems taking bibliometric approach. Four research questions are discussed: (1) the inter-correlation among different ranking systems; (2) the intra-correlation within ranking systems; (3) the correlation of indicators across ranking systems; and (4) the impact of different citation indexes on rankings. The preliminary results show that 55?% of top 200 universities are covered in all ranking systems. The rankings of ARWU and PRSPWU show stronger correlation. With inclusion of another ranking, WRWU (2009?C2010), these rankings tend to converge. In addition, intra-correlation is significant and this means that it is possible to find out some ranking indicators with high degree of discriminativeness or representativeness. Finally, it is found that there is no significant impact of using different citation indexes on the ranking results for top 200 universities.     

A critical comparative analysis of five world university rankings

Scientometrics - To provide users insight into the value and limits of world university rankings, a comparative analysis is conducted of five ranking systems: ARWU, Leiden, THE, QS and U-Multirank....     

Irreproducibility of the results of the Shanghai academic ranking of world universities

I discuss the difficulties that I encountered in reproducing the results of the Shanghai ranking of world universities. In the Shanghai ranking, the dependence between the score for the SCI indicator and the weighted number of considered articles obeys a power law, instead of the proportional dependence that is suggested by the official methodology of the ranking. Discrepancies from proportionality are also found in some of the scores for the N&S and Size indicators. This shows that the results of the Shanghai ranking cannot be reproduced, given raw data and the public methodology of the ranking.     

Uncovering excellence in academic rankings: a closer look at the Shanghai ranking

In this paper, we examine whether the quality of academic research can be accurately captured by a single aggregated measure such as a ranking. With Shanghai University’s Academic Ranking of World Universities as the basis for our study, we use robust principal component analysis to uncover the underlying factors measured by this ranking. Based on a sample containing the top 150 ranked universities, we find evidence that, for the majority of these institutions, the Shanghai rankings reflect not one but in fact two different and uncorrelated aspects of academic research: overall research output and top-notch researchers. Consequently, the relative weight placed upon these two factors determines to a large extent the final ranking.     

Reproducibility of the Shanghai academic ranking of world universities results

This paper discusses and copes with the difficulties that arise when trying to reproduce the results of the Shanghai academic ranking of world universities. In spite of the ambiguity of the methodology of the ranking with regard to the computation of the scores on its six indicators, the paper presents a set of straightforward procedures to estimate raw results and final relative scores. Discrepancies between estimated scores and the results of the ranking are mostly associated with the difficulties encountered in the identification of institutional affiliations, and are not significant. We can safely state that the results of the Shanghai academic ranking of world universities are in fact reproducible.     

Comparing university rankings

Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.     

Triple helix in the science and technology innovation centers of China from the perspective of mutual information: a comparative study between Beijing and Shanghai

Scientometrics - In this study, we measure the systemness of the triple helix (TH) relationship of universities, industries, and governments (UIGs) of Beijing and Shanghai science and technology...     

South African universities in world rankings

South Africa has 23 universities, of which five are placed in one or more of the 2011 Shanghai Jiao Tong, Times Higher Education, and Quacquarelli Symonds world university rankings. The five are: Cape Town, Witwatersrand, KwaZulu-Natal, Stellenbosch and Pretoria. They are ranked above the other 18 universities, with Cape Town in top position, mainly because they have significantly higher publication and citation counts. In the Shanghai Jiao Tong ranking Cape Town??s Nobel Prize alumni and highly-cited researchers give it an additional lead over second-placed Witwatersrand, which has Nobel Prize alumni but no highly-cited researchers. KwaZulu-Natal, in third place, has no Nobel Prize alumni but one highly-cited researcher, which places it ahead of Stellenbosch and Pretoria despite the latter two having higher publication output. However, in the Times Higher Education ranking, which places Cape Town first and Witwatersrand second, Stellenbosch is ranked but not KwaZulu-Natal, presumably because the publication and citation counts of Stellenbosch are higher. The other 18 universities are ranked by the SCImago and Webometrics rankings in an order consistent with bibliometric indicators, and consistent with approximate simulations of the Shanghai Jiao Tong and Times Higher Education methods. If a South African university aspires to rise in the rankings, it needs to increase publications, citations, staff-student ratio, and proportions of postgraduate students, international students and international staff.     

Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review

Scientometrics - This integrative review reports on methodological questions about the Shanghai Ranking as a tool for the evaluation of universities, questions that are extensible to other...     

A bibliometric tale of two cities: Hong Kong and Singapore

A research evaluation of the leading research universities globally using the curated bibliometric data from Shanghai Rankings Global Ranking of Academic Subjects 2018 shows Hong Kong at the top position. A X/GDP productivity criteria is used where each country’s scientific wealth is represented by X, a second-order bibliometric indicator, and its nominal Gross Domestic Product (GDP) is taken as a measure of its economic wealth. Singapore appears at the fifth position. This brief communication looks at the anatomy of this exceptional performance of the two city states, which are similar in many ways and yet dissimilar in many other ways. Hong Kong has the slight edge over Singapore in translating economic wealth to scientific wealth.

     

Mission impossible? Entrepreneurial universities and peripheral regional innovation systems

As part of their ‘third mission’ to commercialise research and cultivate growth in local economies, universities have been accorded a central role in regional innovation systems. This paper takes issue with this policy emphasis. It presents empirical evidence suggesting the entrepreneurial spillovers from universities have been greatly exaggerated, especially in some peripheral regions. The explanation offered for this poor performance hinges on the substantive disconnect between universities and their surrounding local entrepreneurial and innovation ecosystems. Despite their marginal economic contribution, the paper claims that ‘policy entrepreneurs’ play a powerful role in cumulatively reinforcing the dominant role of universities through a process of ‘institutional capture’, the outcome of which results in a form of ‘policy lock-in’. The implications of these findings for public policy are outlined.     

A reassessment of Asian pacific excellence programs in higher education: the Taiwan experience

With the growth of competition between nations in our knowledge-based world economy, excellence programs are becoming a national agenda item in developing as well as developed Asian countries. The main purpose of this paper is to compare the goals, funding policies and selection criteria of excellence programs in China, Japan, Korea and Taiwan and to analyze the academic achievement of their top ranked universities in three areas: research output, internationalization, and excellence, by using data from the Shanghai Jiao Tong, QS, and HEEACT rankings. The effectiveness of Taiwan??s ??Development Plan for World Class Universities and Research Centers of Excellence?? was assessed as a case study in the paper via a survey targeting on 138 top administrators from 11 Taiwan??s universities and 30 reviewers. The study found that more funding nations had, the more outputs and outcomes they would gain, for example China. The Taiwan case demonstrates that world-class universities and research centers are needed in Asian nations despite the concerns for inequality which they raise.