Adjusted sum of institutional scores as an indicator of the presence of university systems in the ARWU ranking

In this article I introduce a new indicator that measures the presence of a higher education system in the Shanghai Jiao Tong Academic Ranking of World Universities (ARWU). First, the benefits of introducing such a measure and the drawbacks associated with the possible choices of the indicator are discussed. To analyze the drawbacks, the sample of countries with presence in ARWU is split into two groups of small and large world’s GDP share. A raw indicator based upon the sum of the scores of all the universities from a country divided by its world’s GDP share shows a noticeable bias in favor of small countries, so a one-way between-groups analysis of variance is conducted to help in canceling the bias. That leads to the introduction of a new aggregate indicator that can be computed in a very simple fashion. A discussion of the performance of higher education systems using this new indicator closes the paper.     

Performance evaluation of research universities in Mainland China,Hong Kong and Taiwan: based on a two-dimensional approach

For a long time, rankings overused in evaluating Chinese universities’ research performance. The relationship between research production and research quality hasn’t been taken seriously in ranking systems. Most university rankings in China put more weight on research production rather than research quality. Recently, the developmental strategy of Chinese universities has shifted from ‘quantity’ to ‘quality’. As a result, a two-dimensional approach was developed in this article to balance ‘quantity’ and ‘quality’. The research production index and the research quality index were produced to locate research universities (RU) from Mainland China, Hong Kong (HK) and Taiwan (TW) in the two-dimensional graph. Fifty-nine RU were classified into three categories according to their locations, which indicated the relevant level of research performance. University of Hong Kong, National Taiwan University, Tsing Hua University and Peking University appeared to be leading universities in research performance. The result showed that the mainland universities were generally of higher research production and lower research quality than HK and TW universities, and proved that the merging tides of Chinese universities enlarged their research production while causing a low level of research quality as well.     

Coexistence of university-industry relations and academic research: Barrier to or incentive for scientific productivity

In this article we analyse whether university-industry relations (UIR) are penalising research activity and inhibiting university researchers’ scientific productivity and, if so, to what extent. The analysis is based on a case study of two Spanish universities. We find that UIR exercise a positive effect on university scientific productivity only when they are based on the development of R&D contracts, and when the funds obtained through these activities do not exceed 15% of the researcher’s total budget. We also find that researchers who combine research and UIR activities obtain higher funding from competitive public sources than that engage only in research. In addition, their average scientific productivity is higher and they achieve higher status within their institutions than those members of faculty who concentrate only on research.     

A fresh approach to evaluating the academic ranking of world universities

The aim of this article is to present new ideas in evaluating Shanghai University’s Academic Ranking of World Universities (ARWU). One issue frequently put forth in various publications is that the Shanghai rankings are sensitive to the relative weight they attribute to each variable. As a possible remedy to this issue, the statistical I-distance method is proposed to be used. Based on a sample containing the top 100 ranked universities, the results show a significant correlation with the official ARWU list. However, some inconsistencies concerning European universities have been noticed and elaborated upon.     

On the internal dynamics of the Shanghai ranking

The Academic Ranking of World Universities (ARWU) published by researchers at Shanghai Jiao Tong University has become a major source of information for university administrators, country officials, students and the public at large. Recent discoveries regarding its internal dynamics allow the inversion of published ARWU indicator scores to reconstruct raw scores for 500 world class universities. This paper explores raw scores in the ARWU and in other contests to contrast the dynamics of rank-driven and score-driven tables, and to explain why the ARWU ranking is a score-driven procedure. We show that the ARWU indicators constitute sub-scales of a single factor accounting for research performance, and provide an account of the system of gains and non-linearities used by ARWU. The paper discusses the non-linearities selected by ARWU, concluding that they are designed to represent the regressive character of indicators measuring research performance. We propose that the utility and usability of the ARWU could be greatly improved by replacing the unwanted dynamical effects of the annual re-scaling based on raw scores of the best performers.     

Does a country’s scientific ‘productivity’ depend critically on the number of country journals indexed?

In this paper, we examine the question whether it is meaningful to talk about the scientific productivity of nations based on indexes like the Science Citation Index or Scopus, when the journal set covered by them keeps changing with time. We hypothesize from the illustrative case of India’s declining productivity in the 1980s which correlated with a fall in its journals indexed, that an apparent increase/decrease in productivity for any country, based on observed change in its share of papers could, in fact, be an effect resulting from the inclusion of more/less journals from the country. To verify our hypothesis we have used SCIMAGO data. We found that for a set of 90 countries, the share of journals regressed on the share of papers gave a linear relationship that explained 80% of the variance. However, we also show that in the case of China’s unusual rise in world scientific productivity (to second rank crossing several other countries), there is yet another factor that needs to be taken into account. We define a new indicator—the JOURNAL PACKING DENSITY (JPD) or average number of papers in journals from a given country. We show that the packing density of Chinese journals has steadily increased over the last few years. Currently, Chinese journals have the highest ‘packing density’ in the world, almost twice the world average which is about 100 papers per journal per annum. The deviation of the JPD from the world average is another indicator which will affect so called ‘national productivities’ in addition to the number of national journals indexed. We conclude that in the context of a five fold increase in the number of journals indexed over 20 years, the simplistic notion of ‘scientific productivity’ as equivalent to papers indexed needs to be re-examined.     

A comparative study on world university rankings: a bibliometric survey

Recently there are many organizations conducting projects on ranking world universities from different perspectives. These ranking activities have made impacts and caused controversy. This study does not favor using bibliometric indicators to evaluate universities?? performances, but not against the idea either. We regard these ranking activities as important phenomena and aim to investigate correlation of different ranking systems taking bibliometric approach. Four research questions are discussed: (1) the inter-correlation among different ranking systems; (2) the intra-correlation within ranking systems; (3) the correlation of indicators across ranking systems; and (4) the impact of different citation indexes on rankings. The preliminary results show that 55?% of top 200 universities are covered in all ranking systems. The rankings of ARWU and PRSPWU show stronger correlation. With inclusion of another ranking, WRWU (2009?C2010), these rankings tend to converge. In addition, intra-correlation is significant and this means that it is possible to find out some ranking indicators with high degree of discriminativeness or representativeness. Finally, it is found that there is no significant impact of using different citation indexes on the ranking results for top 200 universities.     

A new methodology for ranking scientific institutions

This paper examines policy-relevant effects of a yearly public ranking of individual researchers and their institutes in economics by means of their publication output in international top journals. In 1980, a grassroots ranking (‘Top 40’) of researchers in the Netherlands by means of their publications in international top journals started a competition among economists. The objective was to improve economics research in the Netherlands to an internationally competitive level. The ranking lists did stimulate output in prestigious international journals. Netherlands universities tended to perform well compared to universities elsewhere in the EU concerning volume of output in ISI source journals, but their citation impact was average. Limitations of ranking studies and of bibliometric monitoring in the field of economics are discussed.     

The measurement of Italian universities’ research productivity by a non parametric-bibliometric methodology

This paper presents a methodology for measuring the technical efficiency of research activities. It is based on the application of data envelopment analysis to bibliometric data on the Italian university system. For that purpose, different input values (research personnel by level and extra funding) and output values (quantity, quality and level of contribution to actual scientific publications) are considered. Our study aims at overcoming some of the limitations connected to the methodologies that have so far been proposed in the literature, in particular by surveying the scientific production of universities by authors’ name.     

How does the normalization of data affect the ARWU ranking?

The aim of this paper is to present new ideas in evaluating Shanghai University??s Academic Ranking of World Universities (ARWU). In particular, this paper shall try to determine whether the normalization of data affects University ranks. In accordance with this, both the normalized and original (raw) data for each of the six variables has been obtained. Based on a sample containing the 54 US universities which are placed in the ARWU top 100, the statistical I-distance method was performed. The results showed great inconsistencies between university ranks obtained for the original and normalized data. These findings were then analyzed and the universities that had the greatest fluctuation in their ranks were noted.     

The role of Chinese universities in enterprise–university research collaboration

In this paper the role of Chinese universities in enterprise–university research collaboration is investigated. This study focuses on a special aspect of the collaboration—co-authored articles. The two cases are analyzed: (1) research collaboration between Baosteel Group Corporation and Chinese universities; (2) research collaboration between China Petroleum & Chemical Corporation and Chinese universities. The co-authorship data over the period 1998–2007 were searched from CNKI database, the largest Chinese publication and citation database. The main findings are as follows: the number of articles co-authored by enterprise and university scientists has been increasing rapidly; the share of co-authored articles has been growing; the authors from universities are more possible to be the first authors; as a whole, enterprise–university co-authored articles tend to receive more citations and get downloaded more frequently; a mathematical orientation emerges in the enterprise–university articles. To reveal and describe such a trend the methods of keywords analysis and co-occurrence analysis are applied. The Chinese government’s policy instruments and substantial supports for pushing and improving enterprise–university research collaboration are introduced and analyzed.     

A methodology for Institution-Field ranking based on a bidimensional analysis: the <Emphasis Type="Italic">IFQ</Emphasis><Superscript>2</Superscript><Emphasis Type="Italic">A</Emphasis> index

The problem of comparing academic institutions in terms of their research production is nowadays a priority issue. This paper proposes a relative bidimensional index that takes into account both the net production and the quality of it, as an attempt to provide a comprehensive and objective way to compare the research output of different institutions in a specific field, using journal contributions and citations. The proposed index is then applied, as a case study, to rank the top Spanish universities in the fields of Chemistry and Computer Science in the period ranging from 2000 until 2009. A comparison with the top 50 universities in the ARWU rankings is also made, showing the proposed ranking is better suited to distinguish among non-elite universities.     

Relations between national research investment and publication output: Application to an American Paradox

The term “European Paradox” describes the perceived failure of the EU to capture full benefits of its leadership of science as measured by publications and some other indicators. This paper investigates what might be called the “American Paradox,” the decline in scientific publication share of the U.S. despite world-leading investments in research and development (R&D) — particularly as that decline has accelerated in recent years. A multiple linear regression analysis was made of which inputs to the scientific enterprise are most strongly correlated with the number of scientific papers produced. Research investment was found to be much more significant than labor input, government investment in R&D was much more significant than that by industry, and government non-defense investment was somewhat more significant than its defense investment. Since the EU actually leads the U.S. in this key component, this could account for gradual loss of U.S. paper share and EU assumption of leadership of scientific publication in the mid-1990s. More recently the loss of U.S. share has accelerated, and three approaches analyzed this phenomenon: (1) A companion paper shows that the SCI database has not significantly changed to be less favorable to the U.S.; thus the decline is real and is not an artifact of the measurement methods. (2) Budgets of individual U.S. research agencies were correlated with overall paper production and with papers in their disciplines. Funding for the U.S. government civilian, non-healthcare sector was flat in the last ten years, resulting in declining share of papers. Funding for its healthcare sector sharply increased, but there were few additional U.S. healthcare papers. While this inefficiency contributes to loss of U.S. share, it is merely a specific example of the general syndrome that increased American investments have not produced increased publication output. (3) In fact the decline in publication share appears to be due to rapidly increasing R&D investments by China, Taiwan, S. Korea, and Singapore. A model shows that in recent years it is a country’s share of world investment that is most predictive of its publication share. While the U.S. has increased its huge R&D investment, its investment share still declined because of even more rapidly increasing investments by these Asian countries. This has likely led to their sharply increased share of scientific publication, which must result in declines of shars of others — the U.S. and more recently, the EU.     

Core-periphery,new clusters,or rising stars?: international scientific collaboration among ‘advanced’ countries in the era of globalization

Assuming the OECD member states as ‘advanced’ nations equipped with basic scientific capacities, the present research addresses the network configuration of these countries in international scientific collaboration and the transformation of this network along with globalization. The result suggests that geographical, linguistic, and economic affinities did not have a meaningful impact on the formation of co-authorship network between ‘advanced’ nations, different from previous research results which claimed their importance on international cooperation. Globalization facilitated by the development of information and transportation technologies was found to influence the co-authorship link between countries, but not to accelerate centralization of the network in the past 15 years. Though the core-periphery pattern still persists, new rising stars, which are Korea and Turkey, have emerged in the co-authorship network among ‘advanced’ nations. These two countries, having a rapid increase in the share of degree centrality from 1995 to 2010, had strategic financial support from the government which stimulated the collaboration between universities and industries and emphasized the development of science and engineering fields.