A comparative study on world university rankings: a bibliometric survey

Recently there are many organizations conducting projects on ranking world universities from different perspectives. These ranking activities have made impacts and caused controversy. This study does not favor using bibliometric indicators to evaluate universities?? performances, but not against the idea either. We regard these ranking activities as important phenomena and aim to investigate correlation of different ranking systems taking bibliometric approach. Four research questions are discussed: (1) the inter-correlation among different ranking systems; (2) the intra-correlation within ranking systems; (3) the correlation of indicators across ranking systems; and (4) the impact of different citation indexes on rankings. The preliminary results show that 55?% of top 200 universities are covered in all ranking systems. The rankings of ARWU and PRSPWU show stronger correlation. With inclusion of another ranking, WRWU (2009?C2010), these rankings tend to converge. In addition, intra-correlation is significant and this means that it is possible to find out some ranking indicators with high degree of discriminativeness or representativeness. Finally, it is found that there is no significant impact of using different citation indexes on the ranking results for top 200 universities.     

Country-specific determinants of world university rankings

This paper examines country-specific factors that affect the three most influential world university rankings (the Academic Ranking of World Universities, the QS World University Ranking, and the Times Higher Education World University Ranking). We run a cross sectional regression that covers 42–71 countries (depending on the ranking and data availability). We show that the position of universities from a country in the ranking is determined by the following country-specific variables: economic potential of the country, research and development expenditure, long-term political stability (freedom from war, occupation, coups and major changes in the political system), and institutional variables, including government effectiveness.     

Comparing university rankings

Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.     

World-wide university rankings: A Scandinavian approach

Although universities’ world rankings are popular, their design and methods still request considerable elaborations. The paper demonstrates some shortcomings in the Academic World Ranking of Universities (ARWU, Shanghai Jiao Tong University) ranking methods. One deficiency is that universities’ scale differences are neglected due to omitting the whole input side. By resampling and reanalyzing the ARWU data, the paper proposes an input-output analysis for measuring universities’ scientific productivity with special emphasis on those universities which meet the productivity threshold (i.e. share of output exceeds share of input) in a certain group of universities. The productivity analysis on Scandinavian universities evaluates multidisciplinary and specialized universities on their own terms; consequently the ranking based on scientific productivity deviates significantly from the ARWU.     

South African universities in world rankings

South Africa has 23 universities, of which five are placed in one or more of the 2011 Shanghai Jiao Tong, Times Higher Education, and Quacquarelli Symonds world university rankings. The five are: Cape Town, Witwatersrand, KwaZulu-Natal, Stellenbosch and Pretoria. They are ranked above the other 18 universities, with Cape Town in top position, mainly because they have significantly higher publication and citation counts. In the Shanghai Jiao Tong ranking Cape Town??s Nobel Prize alumni and highly-cited researchers give it an additional lead over second-placed Witwatersrand, which has Nobel Prize alumni but no highly-cited researchers. KwaZulu-Natal, in third place, has no Nobel Prize alumni but one highly-cited researcher, which places it ahead of Stellenbosch and Pretoria despite the latter two having higher publication output. However, in the Times Higher Education ranking, which places Cape Town first and Witwatersrand second, Stellenbosch is ranked but not KwaZulu-Natal, presumably because the publication and citation counts of Stellenbosch are higher. The other 18 universities are ranked by the SCImago and Webometrics rankings in an order consistent with bibliometric indicators, and consistent with approximate simulations of the Shanghai Jiao Tong and Times Higher Education methods. If a South African university aspires to rise in the rankings, it needs to increase publications, citations, staff-student ratio, and proportions of postgraduate students, international students and international staff.     

Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings

We applied a set of standard bibliometric indicators to monitor the scientific state-of-arte of 500 universities worldwide and constructed a ranking on the basis of these indicators (Leiden Ranking 2010). We find a dramatic and hitherto largely underestimated language effect in the bibliometric, citation-based measurements of research performance when comparing the ranking based on all Web of Science (WoS) covered publications and on only English WoS covered publications, particularly for Germany and France.     

Institutional research rankings via bibliometric analysis and direct peer review: A comparative case study with policy implications

Recent years have seen enormously increased interest in the comparative evaluation of research quality in the UK, with considerable resources devoted to ranking the output of academic institutions relative to one another at the sub-discipline level, and the disposition of even greater resources dependent on the outcome of this process. The preferred methodology has been that of traditional peer review, with expert groups of academics tasked to assess the relative worth of all research activity in ‘their’ field. Extension toinstitutional evaluation of a recently refined technique ofjournal ranking (Discipline Contribution Scoring) holds out the possibility of ‘automatic’ evaluation within a time-frame considerably less than would be required using methods based directly on citation counts within the corpus of academic work under review. This paper tests the feasibility of the technique in the sub-field of Business and Management Studies Research, producing rankings which are highly correlated with those generated by the much more complex and expensive direct peer review approach. More generally, the analysis also gives a rare opportunity directly to compare the equivalence of peer review bibliometric analysis over a whole sub-field of academic activity in a non-experimental setting.     

Self-defined information indices: application to the case of university rankings

Scientometrics - University rankings are now relevant decision-making tools for both institutional and private purposes in the management of higher education and research. However, they are often...     

An insight into the importance of national university rankings in an international context: the case of the I-UGR rankings of Spanish universities

The great importance international rankings have achieved in the research policy arena warns against many threats consequence of the flaws and shortcomings these tools present. One of them has to do with the inability to accurately represent national university systems as their original purpose is only to rank world-class universities. Another one has to do with the lack of representativeness of universities’ disciplinary profiles as they usually provide a unique table. Although some rankings offer a great coverage and others offer league tables by fields, no international ranking does both. In order to surpass such limitation from a research policy viewpoint, this paper analyzes the possibility of using national rankings in order to complement international rankings. For this, we analyze the Spanish university system as a study case presenting the I-UGR Rankings for Spanish universities by fields and subfields. Then, we compare their results with those obtained by the Shanghai Ranking, the QS Ranking, the Leiden Ranking and the NTU Ranking, as they all have basic common grounds which allow such comparison. We conclude that it is advisable to use national rankings in order to complement international rankings, however we observe that this must be done with certain caution as they differ on the methodology employed as well as on the construction of the fields.     

Applying the CSS method to bibliometric indicators used in (university) rankings

Scientometrics - This Letter to the Editor proposes to use the CSS method for classifying ranking results (e.g. from university rankings) into meaningful groups.     

A comparative analysis of a-C:H films deposited from five hydrocarbons by thermal desorption spectroscopy

Thermal induced gas evolution studies were performed on hydrogenated amorphous carbon (a-C:H) films deposited by plasma enhanced chemical vapour deposition (PECVD). The hydrocarbon-argon discharges were powered by asymmetric bipolar voltage pulses. Five different hydrocarbons (acetylene C₂H₂, isobutene C₄H₈, cyclopentene C₅H₈, toluene C₇H₈ and cycloheptatriene C₇H₈) were compared as film growth precursors.The films were heated in vacuum upto 1000 °C and at the same time the gaseous effusion products were measured with a quadrupole mass spectrometer (QMS). The release of hydrogen, water, argon and different hydrocarbons from CH₄ upto C₇H₈ was proved. Large hydrocarbons were detected only when heating up soft films. When annealing hard films, they almost only lost hydrogen - mainly as H₂ but also in remarkable portion as H₂O. Possibly, the water in the a-C:H films contains the main fraction of hydrogen unbounded to carbon.The characteristic temperatures for the onset of gas evolution of the molecular gases and of argon were found to be strictly correlated with the film hardness - independent of the film growth precursor. The threshold temperature for the release of the dominating effusion product, molecular hydrogen, is significantly higher than that of argon only for films with hardness below about 13 GPa.     

A national-scale cross-time analysis of university research performance

Research policies in the more developed nations are ever more oriented towards the introduction of productivity incentives and competition mechanisms intended to increase efficiency in research institutions. Assessments of the effects of these policy interventions on public research activity often neglect the normal, inherent variation in the performance of research institutions over time. In this work, we propose a cross-time bibliometric analysis of research performance by all Italian universities in two consecutive periods (2001–2003 and 2004–2008) not affected by national policy interventions. Findings show that productivity and impact increased at the level of individual scientists. At the level of university, significant variation in the rank was observed.     

A comparative study on five approaches to evaluate double-K fracture toughness parameters of concrete and size effect analysis

The current paper presents a comprehensive comparison of double-K fracture toughness parameters of concrete evaluated using experimental method and four existing analytical methods. Fracture tests were carried out on compact tension wedge splitting specimens with various depths varying from 200 mm up to 1000 mm. In the analytical calculation, depending on the relationship between critical crack tip opening displacement and the abscissa value of turning point on bilinear softening curve, two different distributions of cohesive stress are considered along crack extension. Results show that four available analytical calculations yield almost the same values of double-K fracture toughness parameters and agree well with those obtained from the experiment, which confirms the consistency of five approaches. Size effect was discussed, including unstable fracture toughness, initiation fracture toughness, critical effective crack length, the length of critical fracture process zone and critical crack tip opening displacement.     

A disaster-severity assessment DSS comparative analysis

This paper aims to provide a comparative analysis of fuzzy rule-based systems and some standard statistical and other machine learning techniques in the context of the development of a decision support system (DSS) for the assessment of the severity of natural disasters. This DSS, which will be referred to as SEDD, has been proposed by the authors to help decision makers inside those Non-Governmental Organizations (NGOs) concerned with the design and implementation of international operations of humanitarian response to disasters. SEDD enables a relatively highly accurate and interpretable assessment on the consequences of almost every potential disaster scenario to be obtained through a set of easily accessible information about that disaster scenario and historical data about similar ones. Thus, although SEDD’s methodology is rather sophisticated, its data requirements are small, which, therefore, enables its use in the context of NGOs and countries requiring humanitarian aid. In this sense, SEDD opposes to some current tools which focuses on one phenomena-one place disaster scenarios (earthquakes in California, hurricanes in Florida, etc.) and/or have extensive and/or technologically sophisticated data requirements (real-time remote sensing information, exhaustive building census, etc.). Moreover, although focused on disaster response, SEDD can also be useful in other phases of disaster management, as disaster mitigation or preparedness. Particularly, the predictive accuracy and interpretability of SEDD fuzzy methodology is compared here in a disaster severity assessment context with those of multiple linear regression, linear discriminant analysis, classification trees and support vector machines. After an extensive validation over the EM-DAT disaster database, it is concluded that SEDD outperforms the methods above in the task of simultaneously providing an accurate and interpretable inference tool for the evaluation of the consequences of disasters.     

A comparative analysis of synthetic genetic oscillators

Synthetic biology is a rapidly expanding discipline at the interface between engineering and biology. Much research in this area has focused on gene regulatory networks that function as biological switches and oscillators. Here we review the state of the art in the design and construction of oscillators, comparing the features of each of the main networks published to date, the models used for in silico design and validation and, where available, relevant experimental data. Trends are apparent in the ways that network topology constrains oscillator characteristics and dynamics. Also, noise and time delay within the network can both have constructive and destructive roles in generating oscillations, and stochastic coherence is commonplace. This review can be used to inform future work to design and implement new types of synthetic oscillators or to incorporate existing oscillators into new designs.     

A comparative analysis of hotspot identification methods

The identification of crash hotspots is the first step of the highway safety management process. Errors in hotspot identification may result in the inefficient use of resources for safety improvements and may reduce the global effectiveness of the safety management process. Despite the importance of using effective hotspot identification (HSID) methods, only a few researchers have compared the performance of various methods. In this research, seven commonly applied HSID methods were compared against four robust and informative quantitative evaluation criteria. The following HSID methods were compared: crash frequency (CF), equivalent property damage only (EPDO) crash frequency, crash rate (CR), proportion method (P), empirical Bayes estimate of total-crash frequency (EB), empirical Bayes estimate of severe-crash frequency (EBs), and potential for improvement (PFI). The HSID methods were compared using the site consistency test, the method consistency test, the total rank differences test, and the total score test. These tests evaluate each HSID method's performance in a variety of areas, such as efficiency in identifying sites that show consistently poor safety performance, reliability in identifying the same hotspots in subsequent time periods, and ranking consistency. To evaluate the HSID methods, five years of crash data from the Italian motorway A16 were used.The quantitative evaluation tests showed that the EB method performs better than the other HSID methods. Test results highlight that the EB method is the most consistent and reliable method for identifying priority investigation locations. The EB expected frequency of total-crashes (EB) performed better than the EB expected frequency of severe-crashes (EBs), although the results differed only slightly when the number of identified hotspots increased. The CF method performed better than other HSID methods with more appealing theoretical arguments. In particular, the CF method performed better than the CR method. This result is quite alarming, since many agencies use the CR method. The PFI and EPDO methods were largely inconsistent. The proportion method performed worst in all of the tests. Overall, these results are consistent with the results of previous studies.The identification of engineering countermeasures that may reduce crashes was successful in all of the hotspots identified with the EB method; this finding shows that the identified hotspots can also be corrected.The advantages associated with the EB method were based on crash data from one Italian motorway, and the relative performances of HSID methods may change when using other crash data. However, the study results are very significant and are consistent with earlier findings. To further clarify the benefits of the EB method, this study should be replicated in other countries. Nevertheless, the study results, combined with previous research results, strongly suggest that the EB method should be the standard in the identification of hotspots.     

A critical analysis of federal highway safety policy

The contention that is examined and documented in the following article is that the most critical issues in highway safety throughout the world are policy-related; that is, solutions are dependent upon policy commitments to achieve an optimum balance between risk, resources and mobility. Emphases on purely technological solutions and upon actions derived from emotional, intuitive policies are leading to large expenditures and disappointing results.

For the 1966–1971 period, increases in expenditures for highway safety in the United States have been in the order of $10–$20 billion. Most of these costs have been borne by the highway user, not through taxation but through executive action based upon the 1966 Highway Safety Acts. While the fatality rate is lower per hundred-million vehicle-miles of travel, the total annual number killed on U.S. highways has increased from 53,000 to 56,000 from 1966 to 1972.

A type of cost effectiveness approach is suggested, based upon specific policy goals in terms of maximum annual fatalities, available resources and desired mobility. It is argued that for a given level of resources, the greatest reduction in fatalities will be derived if funds are used to implement corrective measures that have the lowest cost per life saved.     

A critical analysis of quadratic beam finite elements

A new methodology of evaluation of C⁰ beam elements is presented. It is shown that, knowing the stiffness matrix of an arbitrary type of element, it is possible to create equivalent equilibrium conditions expressed in the form of one difference equation for a regular beam discretized by these elements. The study of the convergence of one difference equation gives an interpretation of the source of troubles occurring in low-order bending elements which is more convincing than the usually applied consideration of the conditioning of element stiffness matrices. A careful examination of quadratic Mindlin elements provides a very clear explanation of the shear locking essence in the Timoshenko beam. The presented method enables one to identify errors that appear also in the reduced integrated or constrained elements. For each type of analysed quadratic element an adequate difference equation is derived and compared with the exact one. Based on this comparison a simple method of corrections is proposed that completely eliminates the errors associated with the application of C⁰ bending elements.     

A bibliometric analysis of world volatile organic compounds research trends

This study explores a bibliometric approach to quantitatively assessing current research trends on volatile organic compounds, by using the related literature in the Science Citation Index (SCI) database from 1992 to 2007. The articles acquired from such literature were concentrated on the general analysis by scientific output, the research performances by countries, institutes, and collaborations, and the research trends by the frequency of author keywords, words in title, words in abstract, and keywords plus. Over the past years, there had been a notable growth trend in publication outputs, along with more participation and collaboration of countries and institutes. Research collaborative papers had shifted from the national inter-institutional to the international collaboration. Benzene, toluene, and formaldehyde were the three kinds of VOCs concerned mostly. Detection and removing, especially by adsorption and oxidation, of VOCs were to be the orientation of all VOCs research in the next few years.