Analysis of bibliometric indicators for individual scholars in a large data set

Citation numbers and other quantities derived from bibliographic databases are becoming standard tools for the assessment of productivity and impact of research activities. Though widely used, still their statistical properties have not been well established so far. This is especially true in the case of bibliometric indicators aimed at the evaluation of individual scholars, because large-scale data sets are typically difficult to be retrieved. Here, we take advantage of a recently introduced large bibliographic data set, Google Scholar Citations, which collects the entire publication record of individual scholars. We analyze the scientific profile of more than 30,000 researchers, and study the relation between the h-index, the number of publications and the number of citations of individual scientists. While the number of publications of a scientist has a rather weak relation with his/her h-index, we find that the h-index of a scientist is strongly correlated with the number of citations that she/he has received so that the number of citations can be effectively be used as a proxy of the h-index. Allowing for the h-index to depend on both the number of citations and the number of publications, we find only a minor improvement.     

Year-based h-type indicators

We propose new variations of the standard and the real-valued (or interpolated) h-index. More precisely, we propose two different types. For the first type, sources are years, and items are either publications, or citations received or citations per publication. The second type makes use of the speed by which citations are received: it is a diffusion speed index.     

The h-index of h-index and of other informetric topics

In this paper we examine the applicability of the concept of h-index to topics, where a topic has index h, if there are h publications that received at least h citations and the rest of the publications on the topic received at most h citations. We discuss methodological issues related to the computation of h-index of topics (denoted h-b index by BANKS [2006]). Data collection for computing the h-b index is much more complex than computing the index for authors, research groups and/or journals, and has several limitations. We demonstrate the methods on a number of informetric topics, among them the h-index.     

Probing the <Emphasis Type="Italic">h</Emphasis>-core: an investigation of the tail–core ratio for rank distributions

The set of citations received by a set of publications consists of citations received by articles in the h-core and citations received by articles in the h-tail. Denoting the cardinalities of these fours sets as C, P, C _H and C _T we introduce the tail-core ratio (C _T/C _H) and show that in practical cases this ratio tends to increase. Introducing further the k-index, defined as k = (C/P)/(C _T/C _H), we show that this index decreases in most practical cases. A power law model is in accordance with these practical observations.     

A scientometrics law about co-authors and their ranking: the co-author core

Rather than “measuring” a scientist impact through the number of citations which his/her published work can have generated, isn’t it more appropriate to consider his/her value through his/her scientific network performance illustrated by his/her co-author role, thus focussing on his/her joint publications, and their impact through citations? Whence, on one hand, this paper very briefly examines bibliometric laws, like the h-index and subsequent debate about co-authorship effects, but on the other hand, proposes a measure of collaborative work through a new index. Based on data about the publication output of a specific research group, a new bibliometric law is found. Let a co-author C have written J (joint) publications with one or several colleagues. Rank all the co-authors of that individual according to their number of joint publications, giving a rank r to each co-author, starting with r = 1 for the most prolific. It is empirically found that a very simple relationship holds between the number of joint publications J by coauthors and their rank of importance, i.e., J ∝ 1/r. Thereafter, in the same spirit as for the Hirsch core, one can define a “co-author core”, and introduce indices operating on an author. It is emphasized that the new index has a quite different (philosophical) perspective that the h-index. In the present case, one focusses on “relevant” persons rather than on “relevant” publications. Although the numerical discussion is based on one “main author” case, and two “control” cases, there is little doubt that the law can be verified in many other situations. Therefore, variants and generalizations could be later produced in order to quantify co-author roles, in a temporary or long lasting stable team(s), and lead to criteria about funding, career measurements or even induce career strategies.     

Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers

This paper focuses the attention on the ch-index, a recent bibliometric indicator similar to the Hirsch (h) index, to evaluate the published research output of a scientist (Ajiferuke and Wolfram, Proceedings of the 12th international conference of the international society for scientometrics and informetrics. Rio de Janeiro, pp. 798–808, 2009). Ch-index is defined as the number such that, for a general group of scientific publications, ch publications are cited by at least ch different citers while the other publications are cited by no more than ch different citers. The basic difference from the classical h is that, according to ch, the diffusion of one author’s publication is evaluated on the basis of the number of different citing authors (or citers), rather than the number of received citations. The goal of this work is to discuss the pros and cons of ch and identify its connection with h. A large sample of scientists in the Quality Engineering/Management field are analyzed so as to investigate the novel indicator’s characteristics. Then, the analysis is preliminarily extended to other scientific disciplines. The most important result is that ch is almost insensitive to self-citations and/or citations made by recurrent citers, and it can be profitably used for complementing h.     

How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations

Although bibliometrics has been a separate research field for many years, there is still no uniformity in the way bibliometric analyses are applied to individual researchers. Therefore, this study aims to set up proposals how to evaluate individual researchers working in the natural and life sciences. 2005 saw the introduction of the h index, which gives information about a researcher’s productivity and the impact of his or her publications in a single number (h is the number of publications with at least h citations); however, it is not possible to cover the multidimensional complexity of research performance and to undertake inter-personal comparisons with this number. This study therefore includes recommendations for a set of indicators to be used for evaluating researchers. Our proposals relate to the selection of data on which an evaluation is based, the analysis of the data and the presentation of the results.     

On a statistical h index

The measurement of the quality of academic research is a rather controversial issue. Recently Hirsch has proposed a measure that has the advantage of summarizing in a single summary statistics the information that is contained in the citation counts of each scientist. From that seminal paper, a huge amount of research has been lavished, focusing on one hand on the development of correction factors to the h index and on the other hand, on the pros and cons of such measure proposing several possible alternatives. Although the h index has received a great deal of interest since its very beginning, only few papers have analyzed its statistical properties and implications. In the present work we propose a statistical approach to derive the distribution of the h index. To achieve this objective we work directly on the two basic components of the h index: the number of produced papers and the related citation counts vector, by introducing convolution models. Our proposal is applied to a database of homogeneous scientists made up of 131 full professors of statistics employed in Italian universities. The results show that while “sufficient” authors are reasonably well detected by a crude bibliometric approach, outstanding ones are underestimated, motivating the development of a statistical based h index. Our proposal offers such development and in particular confidence intervals to compare authors as well as quality control thresholds that can be used as target values.     

The importance of collaborative networks in Canadian scientific research

Abstract

This study investigates co-author and co-inventor collaborations using scientific articles and patents to measure collaborative knowledge production. This paper assesses how a scientist’s position within the joint co-publication and co-invention network affects its production and citation impact. Our findings reveal that number of publications is strongly associated with the scientists’ position in co-author/inventor networks and that a scientist’s technological production actually increases with collaboration in such networks. These academic relationships have a significant impact on the future number of publication citations and appear to benefit the number of patent citations in the same measure.     

Dentistry scientometric analysis: a comparative study between Brazil and other most productive countries in the area

This research aims at performing a comparative study between the Brazilian scientific production in Dentistry, from 2000 to 2009 and countries that contribute with at least 2 % of the world’s scientific production indexed in the Scopus database. More specifically, we intend to assess the annual Brazilian scientific production by comparing it to the other countries’, analyze the Brazilian and other countries’ publications in journals with higher impact factors, as well as to highlight the scientific production from these countries and its international visibility, measured by its total and by its average of citations and normalized citation index per year, by comparing the countries, and to compare the index h of such countries. As work procedure, the SCImago Journal and Country Rank was used as source, identifying the group of producing countries in the Dentistry area from 1996 to 2009. From a total of 136 countries, 13 were highlighted as the most productive, each one of them accounting for at least 2 % the worldwide scientific production in the area. The following indicators were raised for each country: number of produced documents, total of citations, self-citations, average of citations per document and index h. We verified that Brazil is the only country in Latin America that is pictured among the most productive ones in the Dentistry area. We observed that Brazil presents a growing visibility and impact in the international scenery, what suggests that its production is constantly consolidating, with Brazilian scientific recognition in the main vehicles of dissemination in the area.     

Correlations between bibliometrics and peer evaluation for all disciplines: the evaluation of Brazilian scientists

This paper correlates the peer evaluations performed in late 2009 by the disciplinary committees of CNPq (a Brazilian funding agency) with some standard bibliometric measures for 55 scientific areas. We compared the decisions to increase, maintain or decrease a scientist’s research scholarship funded by CNPq. We analyzed these decisions for 2,663 Brazilian scientists and computed their correlations (Spearman rho) with 21 different measures, among them: total production, production in the last 5 years, production indexed in Web of Science and Scopus, total citations received (according to WOS, Scopus, and Google Scholar), h-index and m-quotient (according to the three citation services). The highest correlations for each area range from 0.95 to 0.29, although there are areas with no significantly positive correlation with any of the metrics.     

Positive and negative aspects of citation indices and journal impact factors

The Hirsch citation index h is nowadays the most frequently used numerical indicator for the performance of scientists as reflected in their output and in the reaction of the scientific community reflected in citations of individual contributions. A few of the possible improvements of h are briefly reviewed. Garfield??s journal impact factor (IF) characterizes the reaction of the scientific community to publications in journals, reflected in citations of all papers published in any given journal during the preceding 2?years, and normalized against all citable articles during the same period. Again, a few of the possible improvements or supplements of IF are briefly reviewed, including the journal-h index proposed by Braun, Gl?nzel, and Schubert. Ascribing higher weighting factors to citations of individual papers proportionally to IF is considered to be a misuse of useful numerical indices based on citations. At most, one could turn this argument on its head and one can find reasons to ascribe an inverse proportionality relative to IF for individual citations: if a paper is considered worthy to be cited even if it was published in a low-IF journal, that citation ought to be worth more than if the citation would have been from a higher-impact journal. A weight factor reflecting the prestige of the citing author(s) may also be considered.     

Publication trends in materials science: a global perspective

This paper attempts to highlight quantitatively and qualitatively the growth and development of world literature on materials science in terms of publication output and citations as per Web of Science (2006–2010). The objective of the study was to perform a scientometric analysis of all materials science research publications in the world. The parameters studied include growth of publications and citations, continent-wise distribution of publications and citations, country-wise distribution of publications, domain-wise distribution of publications and citations, publication efficiency index, distribution of publications and citations according to number of collaborating countries, variation of mean impact factor in materials science domains, identification of highly cited publications and highly preferred journals, quality of research output and application of Bradford’s law.     

Predicting scientific impact based on <Emphasis Type="Italic">h</Emphasis>-index

Predicting the future impact of a scientist/researcher is a critical task. The objective of this work is to evaluate different h-index prediction models for the field of Computer Science. Different combinations of parameters have been identified to build the model and applied on a large data set taken from Arnetminer comprised of almost 1.8 million authors and 2.1 million publications’ record of Computer Science. Machine learning prediction technique, regression, is used to find the best set of parameters suitable for h-index prediction for the scientists from all career ages, without enforcing any constraint on their current h-index values with R ² as a metric to measure the accuracy. Further, these parameters are evaluated for different career ages and different thresholds for h-index values. Prediction results for 1 year are really good, having R ² 0.93 but for 5 years R ² declines to 0.82 on average. Hence inferred that prediction of h-index is difficult for longer periods. Predictions for the researchers having 1 year experience are not precise, having R ² 0.60 for 1 year and 0.33 for 5 years. Considering scientists of different career ages, average R ² values for researchers having 20–36 years of experience were 0.99. For the researches having different h-index values, researchers having low h-index were difficult to predict. Parameters set comprising of current h-index, average citations per paper, number of coauthors, years since publishing first article, number of publications, number of impact factor publications, and number of publications in distinct journals performed better than all other combinations.     

The conundrum of research productivity: a study on sociologists in Italy

This paper aims to understand the influence of institutional and organisational embeddedness on research productivity of Italian sociologists. We looked at all records published by Italian sociologists in Scopus from 1973 to 2016 and reconstructed their co-authorship patterns. We built an individual productivity index by considering the number and type of records, the impact factor of journals in which these records were published and each record’s citations. We found that sociologists who co-authored more frequently with international authors were more productive and that having a stable group of co-authors had a positive effect on the number of publications but not on citations. We found that organisational embeddedness has a positive effect on productivity at the group level (i.e., sociologists working in the same institute), less at the individual level. We did not found any effect of the scientific disciplinary sectors, which are extremely influential administratively and politically for promotion and career in Italy. With all caveats due to several limitations of our analysis, our findings suggest that internationalisation and certain context-specific organisational settings could promote scientist productivity .     

Soil science and the <Emphasis Type="Italic">h</Emphasis> index

Soil science is a relatively young and specialised field of science. This note discusses the use of the h index as a scientific output measure in soil science. We explore the governing factors of h index in soil science: the number of soil scientists, the number of papers published, the average number of citations, and the age of the scientist. We found the average relationship between h index and scientific age for soil science: h = 0.7 t. The h index for soil science is smaller than other major science disciplines but norms for h need to be established     

Bibliometric analysis of biotechnology research in Pakistan

In this study we present an analysis of the research trends in Pakistan in the field of biotechnology for the period 1980–2011. Starting with just 15 publications in 1980 with a negligible annual growth rate for the initial 15 years, the number of publications reached 3,273 in 2011 with an annual growth rate of 22 % for the last 15 years. This growth in publications is studied through factors such as Relative Growth Rate and Doubling Time. A comparison of organizations actively engaged in research in biotechnology is made through factors such as their total publications, total citations, and average citations per paper and indices that determine the quality of publications like h-index, g-index, hg-index and p-index. University of Karachi shows the highest number of publications (2,698), while National Institute of Biotechnology and Genetic Engineering with fewer publications shows the highest average citation per paper (8.07). Agha Khan University however, shows the highest h, g, hg and p indices.     

Does the <Emphasis Type="Italic">h</Emphasis>-index for ranking of scientists really work?

Summary Hirsch (2005) has proposed the h-index as a single-number criterion to evaluate the scientific output of a researcher (Ball, 2005): A scientist has index h if h of his/her N_p papers have at least h citations each, and the other (N_p − h) papers have fewer than h citations each. In a study on committee peer review (Bornmann & Daniel, 2005) we found that on average the h-index for successful applicants for post-doctoral research fellowships was consistently higher than for non-successful applicants.     

On the relationship between citations of publication output and Hirsch index h of authors: conceptualization of tapered Hirsch index h T, circular citation area radius R and citation acceleration a

The nature of the empirical proportionality constant A in the relation L = Ah ² between total number of citations L of the publication output of an author and his/her Hirsch index h is analyzed using data of the publication output and citations for six scientists elected to the membership of the Royal Society in 2006 and 199 professors working in different institutions in Poland. The main problem with the h index of different authors calculated by using the above relation is that it underestimates the ranking of scientists publishing papers receiving very high citations and results in high values of A. It was found that the value of the Hirsch constant A for different scientists is associated with the discreteness of h and is related to the tapered Hirsch index h _T by A ^1/2 ≈ 1.21h _T. To overcome the drawback of a wide range of A associated with the discreteness of h for different authors, a simple index, the radius R of circular citation area, defined as R = (L/π)^1/2 ≈ h, is suggested. This circular citation area radius R is easy to calculate and improves the ranking of scientists publishing high-impact papers. Finally, after introducing the concept of citation acceleration a = L/t ² = π(R/t)² (t is publication duration of a scientist), some general features of citations of publication output of Polish professors are described in terms of their citability. Analysis of the data of Polish professors in terms of citation acceleration a shows that: (1) the citability of the papers of a majority of physics and chemistry professors is much higher than that of technical sciences professors, and (2) increasing fraction of conference papers as well as non-English papers and engagement in administrative functions of professors result in decreasing citability of their overall publication output.     

The H l -index: improvement of H-index based on quality of citing papers

This paper proposes h _l -index as an improvement of the h-index, a popular measurement for the research quality of academic researchers. Although the h-index integrates the number of publications and the academic impact of each publication to evaluate the productivity of a researcher, it assumes that all papers that cite an academic article contribute equally to the academic impact of this article. This assumption, of course, could not be true in most times. The citation from a well-cited paper certainly brings more attention to the article than the citation from a paper that people do not pay attention to. It therefore becomes important to integrate the impact of papers that cite a researcher’s work into the evaluation of the productivity of the researcher. Constructing a citation network among academic papers, this paper therefore proposes h _l -index that integrating the h-index with the concept of lobby index, a measures that has been used to evaluate the impact of a node in a complex network based on the impact of other nodes that the focal node has direct link with. This paper also explores the characteristics of the proposed h _l -index by comparing it with citations, h-index and its variant g-index.