Rankings of universities in the FRG

The following article examines whether an aggregate comparison (i.e. without discriminating by subject) of university performance in the FRG resulting in a ranking of universities is feasible. First, methods of efficiency measurement are reviewed and possible indicators discussed. In the next part, five indicators are extracted for empirical analysis from a catalogue of ten indicators. Even these have to be used carefully. The last chapter presents results of the analysis. In short, the following conclusions can be drawn: 1. Of the five indicators, none represents a single comprehensive measure of research performance. 2. An aggregate measurement of university research performance, if feasible at all, has to be carried out separately for institutes of technology, universities and comprehensive institutions. 3. Even then, a number of serious statistical problems arise in regard to the methods subsequently applied.     

Citation-based criteria of the significance of the research activity of scientific teams

The objective of this research is elaborating new criteria for evaluating the significance of the research results achieved by scientific teams. It is known, that the h-index (Hirsch index) is used to evaluate scientific organizations, as well as individual scientific workers. On the one hand, such a scientometric indicator as the “h-index of a scientific organization” reflects the organization’s scientific potential objectively. On the other hand, it does not always adequately reflect the significance that the results of a scientific team’s research activity have for the scientific megaenvironment (scientific community). The i-index has an even greater disadvantage, being principally limited by the size of a scientific team, although h-index is also dependent on the number of publications. Not trying to diminish the significance of the traditional parameters for monitoring the research activity of scientific organizations, including the institutions of higher education, the authors stress the necessity of using not only the traditional indicators, but also other parameters reflecting the significance of a scientific team’s research results for the scientific community. It should also not be forgotten that a scientific team is a social system whose functioning is not limited to the “sum” of individual scientific workers’ activities. The authors suggest new criteria of significance of research activity of scientific teams, which are suitable for the specific usage, hence they (the indicators) should be used with great caution; it is most appropriate to use the authors’ criteria for analyzing the dynamics of the research activity of scientific teams (following the principle “Compare yourself with yesterday’s yourself”). The authors’ proposed citation-based indicators make it possible to evaluate the true significance of research activity of a scientific team for the scientific community; while defining and justifying the new criteria, the authors also took into consideration the actuality of such a problem as the struggle with the self-citation effect (in a wider context—the problem of struggling with the artificial “improvement” of the scientometric indicators). The methodological basis of the research is formed by the system, metasystem, probability statistic, synergetic, sociological and qualimetric approaches. The research methods are the analysis of the problem situation, the analysis of the scientific literature and the best practices of research activity management at the institutions of higher education (benchmarking), the cognitive, structural–functional and mathematical modelling, the methods of graph, set and relation theory, the methods of qualimetry (the theory of latent variables), the methods of probability theory and mathematical statistics.     

Impact of bibliometrics upon the science system: Inadvertent consequences?

Summary Ranking of research institutions by bibliometric methods is an improper tool for research performance evaluation, even at the level of large institutions. The problem, however, is not the ranking as such. The indicators used for ranking are often not advanced enough, and this situation is part of the broader problem of the application of insufficiently developed bibliometric indicators used by persons who do not have clear competence and experience in the field of quantitative studies of science. After a brief overview of the basic elements of bibliometric analysis, I discuss the major technical and methodological problems in the application of publication and citation data in the context of evaluation. Then I contend that the core of the problem lies not necessarily at the side of the data producer. Quite often persons responsible for research performance evaluation, for instance scientists themselves in their role as head of institutions and departments, science administrators at the government level and other policy makers show an attitude that encourages 'quick and dirty' bibliometric analyses whereas better quality is available. Finally, the necessary conditions for a successful application of advanced bibliometric indicators as support tool for peer review are discussed.     

Some observations on the use of bibliometric indicators in the assignment of university chairs

In their article, “Can the Assignment of University Chairs Be Automated?”,Nicolini, Vakula, Balla, andGandini describe the results of their initial attempts at using multiple bibliometric indicators in order to eventually automate the assignment of University chairs at the full and associate professor levels. The indicators utilized consist of each candidate's age, years of scientific activity, number of published articles, citation rate and the quality and type of publishing and citing journals. Data concerning these indicators are obtained from both SCI databases and the curriculum vitae of 76 test-candidates. Although the ranking of candidates is shown to be affected by both subfield differences in citation patterns and the weighting factor assigned to the bibliometric indicators tested, some predictive validity is found between the use bibliometric indicators and the independent peer-review of candidates. While these results are encouraging, the authors readily acknowledge the limitations of their efforts and the need to further refine the use of bibliometric indicators, before their employment in the automated assignment of University chairs. My observations concerning the work ofNicolini et al. will take the following form. First, I wish to comment briefly on what I view to be the philosophy behind the efforts ofNicolini et al. Second, I want to mention several positive and potentially negative procedural issues associated with the proposed use of bibliometric indicators in assessing individual scientific accomplishments. And finally, I would like to comment on what I see as some broader, latent consequences, potentially associated with the use of bibliometric measures in the automated assignment of University chairs.     

Inconsistent transliteration of Iranian university names: a hazard to Iran’s ranking in ISI Web of Science

Today, university ranking has turned into a critical issue in the world. Each university is identified with a surface form under which the whole performance of that university is assessed. This article intends to provide a clear picture of the inconsistencies observed in recording Iranian university titles by their affiliated authors and to clarify the negative impact of such inconsistencies in positioning Iranian universities in global university ranking systems. To collect various surface forms of Iranian university names, use was made of ISI Web of Science through keywords Cu = Iran and py = 2000–2009. Only MSRT universities were considered. Two M.A. experts listed all variant forms of a single university under that name. The form publicized in a university’s website was considered as its entry name. The major sources of variation identified were as follows: Acronyms, misspellings, abbreviations, space variations, syntactic permutation, application of vowels/consonants and vowel/consonant combinations, /a/vs./aa/, Tashdid, Kasra ezafe, redundancy, downcasing, voiceless glottal stop sound /?/, shortening and deletion of titles. It was found that at its present shape Iranian universities are not receiving the rank they really deserve simply because authors affiliated to a university use university title forms inconsistently. It was recommended that authors follow the surface form publicized by universities in their websites, use the help of an editor in their works, and not be credited for their articles in case the forms deviate from those publicized through the websites. A spell checker, as an add-ins software is highly needed to homogenize Iranian university surface forms by replacing the variants by the dominant form proposed.     

What do global university rankings really measure? The search for the X factor and the X entity

Most academic rankings attempt to measure the quality of university education and research. However, previous studies that examine the most influential rankings conclude that the variables they use could be an epiphenomenon of an X factor that has little to do with quality. The aim of this study is to investigate the existence of this hidden factor or profile in the two most influential global university rankings in the world: the Academic Ranking of World Universities (ARWU) of the University of Shanghai Jiao Tong, and the Times Higher Education (THE) ranking. Results support the existence of an underlying entity profile, characterized by institutions normally from the US that enjoy a high reputation. Results also support the idea that rankings lack the capacity to assess university quality in all its complexity, and two strategies are suggested in relation to the vicious circle created between institutional reputation and rankings.     

Geographical knowledge diffusion and spatial diversity citation rank

This paper proposes a citation rank based on spatial diversity (SDCR) in terms of cities and countries, focusing on the measurement of the “spatial” aspect in citation networks. Our main goal is to solve the citation bias caused by different geographical locations of citations. We empirically investigate spatial properties of citing distances, citation patterns and spatial diversity to understand geographical knowledge diffusion, based on the data from “Transportation Science and Technology” subject category in the Web of Science (1966–2009). We also compare the proposed ranking method with other bibliometric measures, and conduct a case study to figure out the recent ranks of the well-established authors in Transportation research. It is found that the SDCR of a focal author is highly correlated with the sum of spatial diversity weights (“strength”) of all his in-links, and it is better to set the damping factors smaller than 0.75 when ranking authors with various initial academic years by SDCR. The cases show that Hong Kong is becoming a cluster in Transportation research.     

Absolute and specific measures of research group excellence

A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly funded research institutions and universities to serve as a basis for their ranking and financing. While citation-based indicators and metrics are easily accessible, they are far from being universally accepted as way to automate or inform evaluation processes or to replace evaluations based on peer review. Here we consider absolute measurements of research excellence at an amalgamated, institutional level and specific measures of research excellence as performance per head. Using biology research institutions in the UK as a test case, we examine the correlations between peer review-based and citation-based measures of research excellence on these two scales. We find that citation-based indicators are very highly correlated with peer-evaluated measures of group strength, but are poorly correlated with group quality. Thus, and almost paradoxically, our analysis indicates that citation counts could possibly form a basis for deciding on, how to fund research institutions, but they should not be used as a basis for ranking them in terms of quality.     

Reflection of co-authorship networks in the Web: Web hyperlinks versus Web visibility rates

About ten years ago a new research field called “webometrics” emerged. Similarities between methods used in webometrics and scientometrics or informetrics are evident from the literature. Are there also similarities between scientometric and Web indicators of collaboration for possible use in technology policy making? Usually, the bibliometric method used to study collaboration is the investigation of co-authorships.In this paper, Web hyperlinks and Web visibility indicators are examined to establish their usefulness as indicators of collaboration and to explore whether similarities exist between Web-based structures and bibliographic structures.Three empirical studies of collaboration between institutions and individual scientists show that hyperlink structures at the Web don’t reflect collaboration structures collected by bibliographic data. However Web visibility indicators of collaboration are different from hyperlinks and can be successfully used as Web indicators of collaboration.     

Quantifying the publication preferences of leading research universities

Research universities have a strong devotion and advocacy for research in their core academic mission. This is why they are widely recognized for their excellence in research which make them take the most renowned positions in the different worldwide university leagues. In order to examine the uniqueness of this group of universities we analyze the scientific production of a sample of them in a 5 year period of time. On the one hand, we analyze their preferences in research measured with the relative percentage of publications in the different subject areas, and on the other hand, we calculate the similarity between them in research preferences. In order to select a set of research universities, we studied the leading university rankings of Shanghai, QS, Leiden, and Times Higher Education (THE). Although the four rankings own well established and developed methodologies and hold great prestige, we choose to use THE because data were readily available for doing the study we had in mind. Having done that, we selected the twenty academic institutions ranked with the highest score in the last edition of THE World University Rankings 2020 and to contrast their impact, we also, we compared them with the twenty institutions with the lowest score in this ranking. At the same time, we extracted publication data from Scopus database for each university and we applied bibliometrics indicators from Elsevier’s SciVal. We applied the statistical techniques cosine similarity and agglomerative hierarchical clustering analysis to examine and compare affinities in research preferences among them. Moreover, a cluster analysis through VOSviewer was done to classify the total scientific production in the four major fields (health sciences, physical sciences, life sciences and social sciences). As expected, the results showed that top universities have strong research profiles, becoming the leaders in the world in those areas and cosine similarity pointed out that some are more affine among them than others. The results provide clues for enhancing existing collaboration, defining and re-directing lines of research, and seeking for new partnerships to face the current pandemic to find was to tackle down the covid-19 outbreak.

     

The scientific impact and partner selection in collaborative research at Korean universities

This study seeks to bridge the gap between scientometrics literature on scientific collaboration and science and technology management literature on partner selection by linking scientists’ collaborator preferences to the marginal advantage in citation impact. The 1981–2010 South Korea NCR (National Citation Report), a subset of the Web of Science that includes 297,658 scholarly articles, was used for this research. We found that, during this period, multi-author scientific articles increasingly dominated single-author articles: multi-university collaboration grew significantly; and the numbers of research publications produced by teams working within a single institution or by a single author diminished. This study also demonstrated that multi-university collaboration produces higher-impact articles when it includes “Research Universities,” that is, top-tier university schools. We also found that elite universities experienced impact degradation of their scientific results when they collaborated with lower-tier institutions, whereas their lower-tier partners gained impact benefits from the collaboration. Finally, our research revealed that Korean universities are unlikely to work with other universities in the same tier. This propensity for cross-tier collaboration can be interpreted as strategic partner selection by lower-tier schools seeking marginal advantage in citation impact.     

The pursuit of academic excellence and business engagement: is it irreconcilable?

Universities currently need to satisfy the demands of different audiences. In light of the increasing policy emphasis on “third mission” activities, universities are attempting to incorporate these into their traditional missions of teaching and research. University strategies to accomplishing its traditional missions are well-honed and routinized, but the incorporation of the third mission is posing important strategic and managerial challenges for universities. This study explores the relationship between university–business collaborations and academic excellence in order to examine the extent to which academic institutions can balance these objectives. Based on data from the UK Research Assessment Exercise 2001 at the level of the university department, we find no systematic positive or negative relationship between scientific excellence and engagement with industry. Across the disciplinary fields reported in the 2001 Research Assessment Exercise (i.e. engineering, hard sciences, biomedicine, social sciences and the humanities) the relationship between academic excellence and engagement with business is largely contingent on the institutional context of the university department. This paper adds to the growing body of literature on university engagement with business by examining this activity for the social sciences and the humanities. Our findings have important implications for the strategic management of university departments and for higher education policy related to measuring the performance of higher education research institutions.     

Combining reference point based composite indicators with data envelopment analysis: application to the assessment of universities

In our knowledge society, where universities are key players, the assessment of higher education institutions should meet the new demands of the present complex environment. This calls for the use of techniques that are able to manage this complexity. In this paper, we propose a novel combination of methodologies, jointly using a multi-criteria reference point scheme and the data envelopment analysis (DEA) for the assessment of universities. This combination allows us to take into account all the aspects regarded as relevant to assess university performance, and use them as outputs in the efficiency analysis. Our findings highlight the convenience to assess the university performance by using both compensatory and non-compensatory schemes. This way, the information provided allows to detect the actions needed to improve the performances of the universities, rather than just giving an overall performance measure. Furthermore, combining the use of composite indicators with the DEA analysis provides a more complete picture of the institutions assessed, allowing universities to check their efficiency and to detect their weaknesses and strengths accordingly. The approach is illustrated using data of 47 Spanish public universities for the academic year, 2016–2017.

     

Refined and generalized hybrid type quasi-3D shear deformation theory for the bending analysis of functionally graded shells

The closed-form solution of a generalized hybrid type quasi-3D higher order shear deformation theory (HSDT) for the bending analysis of functionally graded shells is presented. From the generalized quasi-3D HSDT (which involves the shear strain functions “f(ζ)” and “g(ζ)” and therefore their parameters to be selected “m” and “n”, respectively), infinite six unknowns' hybrid shear deformation theories with thickness stretching effect included, can be derived and solved in a closed-from. The generalized governing equations are also “m” and “n” parameter dependent. Navier-type closed-form solution is obtained for functionally graded shells subjected to transverse load for simply supported boundary conditions. Numerical results of new optimized hybrid type quasi-3D HSDTs are compared with the first order shear deformation theory (FSDT), and other quasi-3D HSDTs. The key conclusions that emerge from the present numerical results suggest that: (a) all non-polynomial HSDTs should be optimized in order to improve the accuracy of those theories; (b) the optimization procedure in all the cases is, in general, beneficial in terms of accuracy of the non-polynomial hybrid type quasi-3D HSDT; (c) it is possible to gain accuracy by keeping the unknowns constant; (d) there is not unique quasi-3D HSDT which performs well in any particular example problems, i.e. there exists a problem dependency matter.     

URAP-TR: a national ranking for Turkish universities based on academic performance

This study describes the basic methodological approach and the results of URAP-TR, the first national ranking system for Turkish universities. URAP-TR is based on objective bibliometric data resources and includes both size-dependent and size-independent indicators that balance total academic performance with performance per capita measures. In the context of Turkish national university rankings, the paper discusses the implications of employing multiple size-independent and size-dependent indicators on national university rankings. Fine-grained ranking categories for Turkish universities are identified through an analysis of ranking results across multiple indicators.     

National peer-review research assessment exercises for the hard sciences can be a complete waste of money: the Italian case

There has been ample demonstration that bibliometrics is superior to peer-review for national research assessment exercises in the hard sciences. In this paper we examine the Italian case, taking the 2001–2003 university performance rankings list based on bibliometrics as benchmark. We compare the accuracy of the first national evaluation exercise, conducted entirely by peer-review, to other rankings lists prepared at zero cost, based on indicators indirectly linked to performance or available on the Internet. The results show that, for the hard sciences, the costs of conducting the Italian evaluation of research institutions could have been completely avoided.     

Certainty equivalent citation: generalized classes of citation indexes

Drawing from the existing literature on risk and inequality measurement, we implement the notion of “certainty equivalent citation” in order (i) to generalize most of the h-type citation indexes (h-, g-, $\tilde{g},$ t-, f-, w-index), and (ii) to highlight the centrality of the decision-maker’s preferences on distributive aspects (concentration aversion) for the ranking of citation profiles. In order to highlight the sensitivity of citation orderings with respect to concentration aversion, an application to both simulated and real citation profiles is presented.     

Octopus affiliations

Although institutional affiliation is not mandatory in scholarly publishing, a new trend of multiple and simultaneous affiliations, which I will call “octopus affiliations” or “octaffiliations” in short, is increasingly noticeable as a distorted consequence of academic ranking and evaluation systems. Institutions offer financial and/or technical supports to authors whose contributions are, in turn, used as a hallmark of productivity, influence, visibility and reputation of their affiliations at national and international levels. While it is understandable that an author can be affiliated with one or two institutions at a time, multiple and simultaneous affiliations, on the other hand, would be hardly realistic.