On the internal dynamics of the Shanghai ranking

The Academic Ranking of World Universities (ARWU) published by researchers at Shanghai Jiao Tong University has become a major source of information for university administrators, country officials, students and the public at large. Recent discoveries regarding its internal dynamics allow the inversion of published ARWU indicator scores to reconstruct raw scores for 500 world class universities. This paper explores raw scores in the ARWU and in other contests to contrast the dynamics of rank-driven and score-driven tables, and to explain why the ARWU ranking is a score-driven procedure. We show that the ARWU indicators constitute sub-scales of a single factor accounting for research performance, and provide an account of the system of gains and non-linearities used by ARWU. The paper discusses the non-linearities selected by ARWU, concluding that they are designed to represent the regressive character of indicators measuring research performance. We propose that the utility and usability of the ARWU could be greatly improved by replacing the unwanted dynamical effects of the annual re-scaling based on raw scores of the best performers.     

Should you believe in the Shanghai ranking?

This paper proposes a critical analysis of the “Academic Ranking of World Universities”, published every year by the Institute of Higher Education of the Jiao Tong University in Shanghai and more commonly known as the Shanghai ranking. After having recalled how the ranking is built, we first discuss the relevance of the criteria and then analyze the proposed aggregation method. Our analysis uses tools and concepts from Multiple Criteria Decision Making (MCDM). Our main conclusions are that the criteria that are used are not relevant, that the aggregation methodology is plagued by a number of major problems and that the whole exercise suffers from an insufficient attention paid to fundamental structuring issues. Hence, our view is that the Shanghai ranking, in spite of the media coverage it receives, does not qualify as a useful and pertinent tool to discuss the “quality” of academic institutions, let alone to guide the choice of students and family or to promote reforms of higher education systems. We outline the type of work that should be undertaken to offer sound alternatives to the Shanghai ranking.     

What do global university rankings really measure? The search for the X factor and the X entity

Most academic rankings attempt to measure the quality of university education and research. However, previous studies that examine the most influential rankings conclude that the variables they use could be an epiphenomenon of an X factor that has little to do with quality. The aim of this study is to investigate the existence of this hidden factor or profile in the two most influential global university rankings in the world: the Academic Ranking of World Universities (ARWU) of the University of Shanghai Jiao Tong, and the Times Higher Education (THE) ranking. Results support the existence of an underlying entity profile, characterized by institutions normally from the US that enjoy a high reputation. Results also support the idea that rankings lack the capacity to assess university quality in all its complexity, and two strategies are suggested in relation to the vicious circle created between institutional reputation and rankings.     

World-wide university rankings: A Scandinavian approach

Although universities’ world rankings are popular, their design and methods still request considerable elaborations. The paper demonstrates some shortcomings in the Academic World Ranking of Universities (ARWU, Shanghai Jiao Tong University) ranking methods. One deficiency is that universities’ scale differences are neglected due to omitting the whole input side. By resampling and reanalyzing the ARWU data, the paper proposes an input-output analysis for measuring universities’ scientific productivity with special emphasis on those universities which meet the productivity threshold (i.e. share of output exceeds share of input) in a certain group of universities. The productivity analysis on Scandinavian universities evaluates multidisciplinary and specialized universities on their own terms; consequently the ranking based on scientific productivity deviates significantly from the ARWU.     

How does the normalization of data affect the ARWU ranking?

The aim of this paper is to present new ideas in evaluating Shanghai University??s Academic Ranking of World Universities (ARWU). In particular, this paper shall try to determine whether the normalization of data affects University ranks. In accordance with this, both the normalized and original (raw) data for each of the six variables has been obtained. Based on a sample containing the 54 US universities which are placed in the ARWU top 100, the statistical I-distance method was performed. The results showed great inconsistencies between university ranks obtained for the original and normalized data. These findings were then analyzed and the universities that had the greatest fluctuation in their ranks were noted.     

Academic ranking of world universities using scientometrics - A comment to the “Fatal Attraction”

Summary The Institute of Higher Education, Shanghai Jiao Tong University published on the web the Academic Ranking of World Universities and attracted wide attentions worldwide. 60% of their criteria are based on the databases using scientometrics. They were aware of all possible technical problems, have gone through clean up processes and made necessary corrections. Highly cited researchers and articles published in Nature and Science were identified one by one and attributed to the correct institutions. They are confident that errors including human ones in their data are less than two percent. They will continue their ranking efforts, improve their ranking methodologies and provide more choices on the ranking lists.     

University patenting activities and their link to the quantity and quality of scientific publications

Integrating data from three independent data sources––USPTO patenting data, Shanghai Jiao Tong University’s Academic Ranking of World Universities (ARWU) and the Times Higher Education Supplement’s World University Ranking (WUR), we examine the possible link between patenting output and the quantity and quality of scientific publications among 281 leading universities world-wide. We found that patenting by these universities, as measured by patents granted by the USPTO, has grown consistently faster than overall US patenting over 1977–2000, although it has grown more slowly over the last 5 years (2000–2005). Moreover, since the mid-1990s, patenting growth has been faster among universities outside North America than among those within North America. We also found that the patenting output of the universities over 2003–2005 is significantly correlated with the quantity and quality of their scientific publications. However, significant regional variations are found: for universities in North America, both the quantity and quality of scientific publications matter, but for European and Australian/NZ universities, only the quantity of publications matter, while for other universities outside North America and Europe/Australia/NZ, only quality of publications matter. We found similar findings when using EPO patenting data instead of USPTO data. Additionally, for USPTO data only, the degree of internationalization of faculty members is found to reduce patenting performance among North American universities, but to increase that of universities outside North America. Plausible explanations for these empirical observations and implications for future research are discussed.     

Structural characteristics of institutional collaboration in North Korea analyzed through domestic publications

Scientometrics - This paper studies the influence of funding on the position reached by the top 300 universities (institutions) of the Quaccarelly Symond World University Ranking 2018. Geographical...     

Uncovering excellence in academic rankings: a closer look at the Shanghai ranking

In this paper, we examine whether the quality of academic research can be accurately captured by a single aggregated measure such as a ranking. With Shanghai University’s Academic Ranking of World Universities as the basis for our study, we use robust principal component analysis to uncover the underlying factors measured by this ranking. Based on a sample containing the top 150 ranked universities, we find evidence that, for the majority of these institutions, the Shanghai rankings reflect not one but in fact two different and uncorrelated aspects of academic research: overall research output and top-notch researchers. Consequently, the relative weight placed upon these two factors determines to a large extent the final ranking.     

Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings

We applied a set of standard bibliometric indicators to monitor the scientific state-of-arte of 500 universities worldwide and constructed a ranking on the basis of these indicators (Leiden Ranking 2010). We find a dramatic and hitherto largely underestimated language effect in the bibliometric, citation-based measurements of research performance when comparing the ranking based on all Web of Science (WoS) covered publications and on only English WoS covered publications, particularly for Germany and France.     

On using the Shanghai ranking to assess the research performance of university systems

We take a new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems. We deal with system aggregates by means of averaging scores taken over a number of institutions from each higher education system according to the Gross Domestic Product of its country. We treat the set of indicators (measures) at the country level as a scale, and investigate its reliability and dimensionality using appropriate statistical tools. After a Principal Component Analysis is performed, a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale, with a first component that explains more than 72% of the variance of the sample under analysis. The percentages of variance of the indicators explained by the first component do shed light on the fact that ARWU is in fact measuring the research quality (both at the individual and collective levels) of a university system. When the second principal component is taken into account, the two principal components contribute to explain more than 90% of the variance. The rotated solution facilitates the interpretation of the components and provides clear and interesting clustering information about the 32 higher education systems under analysis.     

A minimum violations ranking method

We present a rating method that, given information on the pairwise comparisons of n items, minimizes the number of inconsistencies in the ranking of those items. Our Minimum Violations Ranking (MVR) Method uses a binary linear integer program (BILP) to do this. We prove conditions when the relaxed LP will give an optimal solution to the original BILP. In addition, the LP solution gives information about ties and sensitivities in the ranking. Lastly, our MVR method makes use of bounding and constraint relaxation techniques to produce a fast algorithm for the linear ordering problem, solving an instance with about one thousand items in less than 10 minutes.     

Improving a School's U.S. News and World Report Ranking

In this paper we analyze the 2002 U.S. News and World Report rankings of universities offering graduate programs in engineering. We show that a reasonably simple model, whose independent variables are derived from the intuitive causal relationships among the various performance sub‐measures, can represent the data. The results can be used to quantify the choice of an efficient policy for improving a school's ranking.     

An insight into the importance of national university rankings in an international context: the case of the I-UGR rankings of Spanish universities

The great importance international rankings have achieved in the research policy arena warns against many threats consequence of the flaws and shortcomings these tools present. One of them has to do with the inability to accurately represent national university systems as their original purpose is only to rank world-class universities. Another one has to do with the lack of representativeness of universities’ disciplinary profiles as they usually provide a unique table. Although some rankings offer a great coverage and others offer league tables by fields, no international ranking does both. In order to surpass such limitation from a research policy viewpoint, this paper analyzes the possibility of using national rankings in order to complement international rankings. For this, we analyze the Spanish university system as a study case presenting the I-UGR Rankings for Spanish universities by fields and subfields. Then, we compare their results with those obtained by the Shanghai Ranking, the QS Ranking, the Leiden Ranking and the NTU Ranking, as they all have basic common grounds which allow such comparison. We conclude that it is advisable to use national rankings in order to complement international rankings, however we observe that this must be done with certain caution as they differ on the methodology employed as well as on the construction of the fields.     

Indicators for a webometric ranking of open access repositories

The Ranking Web of World Repositories () is introduced. The objective is to promote Open access initiatives (OAI) supporting the use of repositories for scientific evaluation purposes. A set of metrics based on web presence, impact and usage is discussed. The Ranking is built on indicators obtained from web search engines following a model close to the Impact Factor one. The activity accounts for a 50% of the index, including number of pages, pdf files and items in Google Scholar database, while the visibility takes into account the external inlinks received by the repository (the other 50%). The Ranking provides the Top 300 repositories from a total of 592 worldwide, with a strong presence of US, German and British institutional repositories and the leadership of the large subject repositories. Results suggest the need to take into consideration other file formats and the usage information, an option is not feasible today.     

Comparing university rankings

Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.     

Adjusted sum of institutional scores as an indicator of the presence of university systems in the ARWU ranking

In this article I introduce a new indicator that measures the presence of a higher education system in the Shanghai Jiao Tong Academic Ranking of World Universities (ARWU). First, the benefits of introducing such a measure and the drawbacks associated with the possible choices of the indicator are discussed. To analyze the drawbacks, the sample of countries with presence in ARWU is split into two groups of small and large world’s GDP share. A raw indicator based upon the sum of the scores of all the universities from a country divided by its world’s GDP share shows a noticeable bias in favor of small countries, so a one-way between-groups analysis of variance is conducted to help in canceling the bias. That leads to the introduction of a new aggregate indicator that can be computed in a very simple fashion. A discussion of the performance of higher education systems using this new indicator closes the paper.     

The comprehensive competitiveness evaluation of American universities in Bridge Engineering

This article explores a method of evaluating the comprehensive competitiveness of American universities in Bridge Engineering, which is beneficial for students’ picking up an ideal university for further study in America. Making use of ESI database, SCI database and EI database as well as the ranking of American universities from U.S. News and World Report, the author evaluates the comprehensive competitiveness of American universities in Bridge Engineering, and then develops the ranking of comprehensive competitiveness of American universities in Bridge Engineering specialty. From the ranking, the author reaches the conclusion that American universities such as University of Illinois-Urbana-Champaign and Georgia Institute of Technology and so on, have comparatively higher international influence and competitiveness in the field of Bridge Engineering.     

Profit margin,process improvement and capacity decisions in global manufacturing

We study two strategies that a company may employ for competing in global markets: high profit margin; and investment in process improvements. The strategy of high profit margin is associated with aggressive investment in new plants worldwide; and the strategy of process improvements is associated with increasing the effective capacity of existing plants, reducing manufacturing cost and increasing the plant's life cycle. Such plant decisions are complicated by country-specific parameters, e.g. tariff rate, tax rate, transportation cost and economic growth rate, which may vary widely from one country to another. We construct a simulation model that uses non-linear relationships among decision variables to explore insights, e.g.: (i) global conditions that would be synergistic with each of the two strategies; (ii) level of investment that would be justified in newly industrialized countries, in relation to the industrially mature countries; and (iii) shifts in investment in time and their relationship to the competitive strategies.     

Ranking Iranian universities: an interpretative structural modeling approach

Ranking is widely considered to be an important tool for evaluating the performance, competitiveness, and success of academic institutions. An appropriate ranking system should evaluate the key missions of the higher education system in a way that helps to improve the leadership goals and activities carried out by the universities. Based on the concepts derived from the Iranian Higher Education Upstream Documents and Measures used internationally for university ranking, this study identifies 21 key measures that can be used in the ranking of Iranian universities. The measures are grouped into five categories: scientific infrastructure, scientific effectiveness, socio-cultural effectiveness, international interactions, and sustainability. Then, using the Interpretative Structural Modeling approach, the researchers develop a coherent rubric for establishing the ranking. The proposed conceptual model focuses primarily on the universities’ contribution to technological and scientific infrastructure, then secondarily on their contribution to scientific advancement and international interactions, and finally at a tertiary level on their socio-cultural effectiveness and sustainability.