Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers

This paper focuses the attention on the ch-index, a recent bibliometric indicator similar to the Hirsch (h) index, to evaluate the published research output of a scientist (Ajiferuke and Wolfram, Proceedings of the 12th international conference of the international society for scientometrics and informetrics. Rio de Janeiro, pp. 798–808, 2009). Ch-index is defined as the number such that, for a general group of scientific publications, ch publications are cited by at least ch different citers while the other publications are cited by no more than ch different citers. The basic difference from the classical h is that, according to ch, the diffusion of one author’s publication is evaluated on the basis of the number of different citing authors (or citers), rather than the number of received citations. The goal of this work is to discuss the pros and cons of ch and identify its connection with h. A large sample of scientists in the Quality Engineering/Management field are analyzed so as to investigate the novel indicator’s characteristics. Then, the analysis is preliminarily extended to other scientific disciplines. The most important result is that ch is almost insensitive to self-citations and/or citations made by recurrent citers, and it can be profitably used for complementing h.     

A survey of Quality Engineering–Management journals by bibliometric indicators

This paper analyses some of the most popular scientific journals in the Quality field from the point of view of three bibliometric indicators: the Hirsch (h) index for journals, the total number of citations and the h‐spectrum. In particular, h‐spectrum is a novel tool based on h, making it possible to (i) identify a reference profile of the typical authors of a journal; (ii) compare different journals; and (iii) provide a rough indication of their ‘bibliometric positioning’ in the scientific community. Results of this analysis can be helpful for guiding potential authors and members of the scientific community in the Quality Engineering/Management area. A large amount of empirical data are presented and discussed. Copyright © 2009 John Wiley & Sons, Ltd.     

Analysis of bibliometric indicators for individual scholars in a large data set

Citation numbers and other quantities derived from bibliographic databases are becoming standard tools for the assessment of productivity and impact of research activities. Though widely used, still their statistical properties have not been well established so far. This is especially true in the case of bibliometric indicators aimed at the evaluation of individual scholars, because large-scale data sets are typically difficult to be retrieved. Here, we take advantage of a recently introduced large bibliographic data set, Google Scholar Citations, which collects the entire publication record of individual scholars. We analyze the scientific profile of more than 30,000 researchers, and study the relation between the h-index, the number of publications and the number of citations of individual scientists. While the number of publications of a scientist has a rather weak relation with his/her h-index, we find that the h-index of a scientist is strongly correlated with the number of citations that she/he has received so that the number of citations can be effectively be used as a proxy of the h-index. Allowing for the h-index to depend on both the number of citations and the number of publications, we find only a minor improvement.     

An impact indicator for researchers

The assessment of individual researchers using bibliometric indicators is more complex than that of a region, country or university. For large scientific bodies, averages over a large number of researchers and their outputs is generally believed to give indication of the quality of the research work. For an individual, the detailed peer evaluation of his research outputs is required and, even this, may fail in the short term to make a final, long term assessment of the relevance and originality of the work. Scientometrics assessment at individual level is not an easy task not only due to the smaller number of publications that are being evaluated, but other factors can influence significantly the bibliometric indicators applied. Citation practices vary widely among disciplines and sub disciplines and this may justify the lack of good bibliometric indicators at individual level. The main goal of this study was to develop an indicator that considers in its calculation some of the aspects that we must take into account on the assessment of scientific performance at individual level. The indicator developed, the h _nf index, considers the different cultures of citation of each field and the number of authors per publication. The results showed that the h _nf index can be used on the assessment of scientific performance of individual researchers and for following the performance of a researcher.     

How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations

Although bibliometrics has been a separate research field for many years, there is still no uniformity in the way bibliometric analyses are applied to individual researchers. Therefore, this study aims to set up proposals how to evaluate individual researchers working in the natural and life sciences. 2005 saw the introduction of the h index, which gives information about a researcher’s productivity and the impact of his or her publications in a single number (h is the number of publications with at least h citations); however, it is not possible to cover the multidimensional complexity of research performance and to undertake inter-personal comparisons with this number. This study therefore includes recommendations for a set of indicators to be used for evaluating researchers. Our proposals relate to the selection of data on which an evaluation is based, the analysis of the data and the presentation of the results.     

Further characterizations of the Hirsch index

The Hirsch index is a number that synthesizes a researcher’s output. It is defined as the maximum number h such that the researcher has h papers with at least h citations each. Four characterizations of the Hirsch index are suggested. The most compact one relies on the interpretation of the index as providing the number of valuable papers in an output and postulates three axioms. One, only cited papers can be valuable. Two, the index is strongly monotonic: if output x has more papers than output y and each paper in x has more citations than the most cited paper in y, then x has more valuable papers than y. And three, the minimum amount of citations under which a paper becomes valuable is different for each paper.     

Soil science and the <Emphasis Type="Italic">h</Emphasis> index

Soil science is a relatively young and specialised field of science. This note discusses the use of the h index as a scientific output measure in soil science. We explore the governing factors of h index in soil science: the number of soil scientists, the number of papers published, the average number of citations, and the age of the scientist. We found the average relationship between h index and scientific age for soil science: h = 0.7 t. The h index for soil science is smaller than other major science disciplines but norms for h need to be established     

Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations

The paper proposes two simple new indexes—k and w—to assess a scientist’s publications record based on citations. The two indexes are superior to the widely used h index (Hirsch, 2005), as they preserve all its valuable characteristics and try to overcome one of its shortcomings, i.e. that it uses only a fraction of the information contained in a scientist’s citations profile and, as a result, it is defined over the set of positive integers and does not show a sufficiently fine ‘granularity’ to allow a fully satisfactory ranking of scientists. This problem is particularly acute in many areas of Social Sciences and Humanities, where scientific productivity and citation practices typically yield fewer citations per paper and, as a consequence, are characterized by ‘structurally’ lower values of the h index. Both the indexes proposed are defined over R⁺, their integer part is equal to the scientist’s h index and they fall in the right-open interval [h, h+1). While the h index is influenced only by part of the citations received by a scientist’s most-cited publications, the k index takes into account all the citations received by her most-cited publications and the w index accounts for the citations received by the entire set of her publications. Variants of the k and w indexes are proposed which consider co-authorship. To show the extent to which the h index and the new indexes proposed may yield different results, they are calculated for 332 professors of economics in Italian universities and the results obtained used to rank Italian university departments.     

The h-index of h-index and of other informetric topics

In this paper we examine the applicability of the concept of h-index to topics, where a topic has index h, if there are h publications that received at least h citations and the rest of the publications on the topic received at most h citations. We discuss methodological issues related to the computation of h-index of topics (denoted h-b index by BANKS [2006]). Data collection for computing the h-b index is much more complex than computing the index for authors, research groups and/or journals, and has several limitations. We demonstrate the methods on a number of informetric topics, among them the h-index.     

A scientometrics law about co-authors and their ranking: the co-author core

Rather than “measuring” a scientist impact through the number of citations which his/her published work can have generated, isn’t it more appropriate to consider his/her value through his/her scientific network performance illustrated by his/her co-author role, thus focussing on his/her joint publications, and their impact through citations? Whence, on one hand, this paper very briefly examines bibliometric laws, like the h-index and subsequent debate about co-authorship effects, but on the other hand, proposes a measure of collaborative work through a new index. Based on data about the publication output of a specific research group, a new bibliometric law is found. Let a co-author C have written J (joint) publications with one or several colleagues. Rank all the co-authors of that individual according to their number of joint publications, giving a rank r to each co-author, starting with r = 1 for the most prolific. It is empirically found that a very simple relationship holds between the number of joint publications J by coauthors and their rank of importance, i.e., J ∝ 1/r. Thereafter, in the same spirit as for the Hirsch core, one can define a “co-author core”, and introduce indices operating on an author. It is emphasized that the new index has a quite different (philosophical) perspective that the h-index. In the present case, one focusses on “relevant” persons rather than on “relevant” publications. Although the numerical discussion is based on one “main author” case, and two “control” cases, there is little doubt that the law can be verified in many other situations. Therefore, variants and generalizations could be later produced in order to quantify co-author roles, in a temporary or long lasting stable team(s), and lead to criteria about funding, career measurements or even induce career strategies.     

More axiomatics for the Hirsch index

The Hirsch index is a number that synthesizes a researcher’s output. It is defined as the maximum number h such that the researcher has h papers with at least h citations each. Woeginger (Math Soc Sci 56: 224–232, 2008a; J Informetr 2: 298–303, 2008b) suggests two axiomatic characterizations of the Hirsch index using monotonicity as one of the axioms. This note suggests three characterizations without adopting the monotonicity axiom.     

Beyond the Durfee square: Enhancing the h-index to score total publication output

An individual’s h-index corresponds to the number h of his/her papers that each has at least h citations. When the citation count of an article exceeds h, however, as is the case for the hundreds or even thousands of citations that accompany the most highly cited papers, no additional credit is given (these citations falling outside the so-called “Durfee square”). We propose a new bibliometric index, the “tapered h-index” (h _T), that positively enumerates all citations, yet scoring them on an equitable basis with h. The career progression of h _T and h are compared for six eminent scientists in contrasting fields. Calculated h _T for year 2006 ranged between 44.32 and 72.03, with a corresponding range in h of 26 to 44. We argue that the h _T-index is superior to h, both theoretically (it scores all citations), and because it shows smooth increases from year to year as compared with the irregular jumps seen in h. Conversely, the original h-index has the benefit of being conceptually easy to visualise. Qualitatively, the two indices show remarkable similarity (they are closely correlated), such that either can be applied with confidence. Sadly, after a long and distinguished career, Peter Killworth died on 28 Jan 2008.     

The stability of the <Emphasis Type="Italic">h</Emphasis>-index

Over the last years the h-index has gained popularity as a measure for comparing the impact of scientists. We investigate if ranking according to the h-index is stable with respect to (i) different choices of citation databases, (ii) normalizing citation counts by the number of authors or by removing self-citations, (iii) small amounts of noise created by randomly removing citations or publications and (iv) small changes in the definition of the index. In experiments for 5,283 computer scientists and 1,354 physicists we show that although the ranking of the h-index is stable under most of these changes, it is unstable when different databases are used. Therefore, comparisons based on the h-index should only be trusted when the rankings of multiple citation databases agree.     

Positive and negative aspects of citation indices and journal impact factors

The Hirsch citation index h is nowadays the most frequently used numerical indicator for the performance of scientists as reflected in their output and in the reaction of the scientific community reflected in citations of individual contributions. A few of the possible improvements of h are briefly reviewed. Garfield??s journal impact factor (IF) characterizes the reaction of the scientific community to publications in journals, reflected in citations of all papers published in any given journal during the preceding 2?years, and normalized against all citable articles during the same period. Again, a few of the possible improvements or supplements of IF are briefly reviewed, including the journal-h index proposed by Braun, Gl?nzel, and Schubert. Ascribing higher weighting factors to citations of individual papers proportionally to IF is considered to be a misuse of useful numerical indices based on citations. At most, one could turn this argument on its head and one can find reasons to ascribe an inverse proportionality relative to IF for individual citations: if a paper is considered worthy to be cited even if it was published in a low-IF journal, that citation ought to be worth more than if the citation would have been from a higher-impact journal. A weight factor reflecting the prestige of the citing author(s) may also be considered.     

On the relationship between citations of publication output and Hirsch index h of authors: conceptualization of tapered Hirsch index h T, circular citation area radius R and citation acceleration a

The nature of the empirical proportionality constant A in the relation L = Ah ² between total number of citations L of the publication output of an author and his/her Hirsch index h is analyzed using data of the publication output and citations for six scientists elected to the membership of the Royal Society in 2006 and 199 professors working in different institutions in Poland. The main problem with the h index of different authors calculated by using the above relation is that it underestimates the ranking of scientists publishing papers receiving very high citations and results in high values of A. It was found that the value of the Hirsch constant A for different scientists is associated with the discreteness of h and is related to the tapered Hirsch index h _T by A ^1/2 ≈ 1.21h _T. To overcome the drawback of a wide range of A associated with the discreteness of h for different authors, a simple index, the radius R of circular citation area, defined as R = (L/π)^1/2 ≈ h, is suggested. This circular citation area radius R is easy to calculate and improves the ranking of scientists publishing high-impact papers. Finally, after introducing the concept of citation acceleration a = L/t ² = π(R/t)² (t is publication duration of a scientist), some general features of citations of publication output of Polish professors are described in terms of their citability. Analysis of the data of Polish professors in terms of citation acceleration a shows that: (1) the citability of the papers of a majority of physics and chemistry professors is much higher than that of technical sciences professors, and (2) increasing fraction of conference papers as well as non-English papers and engagement in administrative functions of professors result in decreasing citability of their overall publication output.     

Are you in top 1% (1‰)?

A new simple bibliometric indicator is based on the number of highly cited papers (as defined by WoS^®). It can be used to assess individuals, journals and universities. Unlike most other citation-based-indicators it equalizes the chances of young scientists (vs. their more experienced colleagues) and of scientists working in less-popular disciplines. The ranking of scientists based on the new indicator is not correlated with the rankings based upon the number of citations or on the Hirsch-index.     

Does the <Emphasis Type="Italic">h</Emphasis>-index for ranking of scientists really work?

Summary Hirsch (2005) has proposed the h-index as a single-number criterion to evaluate the scientific output of a researcher (Ball, 2005): A scientist has index h if h of his/her N_p papers have at least h citations each, and the other (N_p − h) papers have fewer than h citations each. In a study on committee peer review (Bornmann & Daniel, 2005) we found that on average the h-index for successful applicants for post-doctoral research fellowships was consistently higher than for non-successful applicants.     

On a statistical h index

The measurement of the quality of academic research is a rather controversial issue. Recently Hirsch has proposed a measure that has the advantage of summarizing in a single summary statistics the information that is contained in the citation counts of each scientist. From that seminal paper, a huge amount of research has been lavished, focusing on one hand on the development of correction factors to the h index and on the other hand, on the pros and cons of such measure proposing several possible alternatives. Although the h index has received a great deal of interest since its very beginning, only few papers have analyzed its statistical properties and implications. In the present work we propose a statistical approach to derive the distribution of the h index. To achieve this objective we work directly on the two basic components of the h index: the number of produced papers and the related citation counts vector, by introducing convolution models. Our proposal is applied to a database of homogeneous scientists made up of 131 full professors of statistics employed in Italian universities. The results show that while “sufficient” authors are reasonably well detected by a crude bibliometric approach, outstanding ones are underestimated, motivating the development of a statistical based h index. Our proposal offers such development and in particular confidence intervals to compare authors as well as quality control thresholds that can be used as target values.     

A publication index that is independent of age

The Hirsch h-index is widely used to measure a researcher’s major publications. It has the advantage of being easy to compute. However, it increases steeply with time and therefore does not allow a comparison of young and mature researchers. We find that if the h-index is divided by the number of decades since publication of the researcher’s first paper, the result is statistically constant with age. Then the resulting index can be compared for young and old researchers. Its accuracy is the same as that of the h-index and is as easy to compute as the h-index.