Positive and negative aspects of citation indices and journal impact factors

The Hirsch citation index h is nowadays the most frequently used numerical indicator for the performance of scientists as reflected in their output and in the reaction of the scientific community reflected in citations of individual contributions. A few of the possible improvements of h are briefly reviewed. Garfield??s journal impact factor (IF) characterizes the reaction of the scientific community to publications in journals, reflected in citations of all papers published in any given journal during the preceding 2?years, and normalized against all citable articles during the same period. Again, a few of the possible improvements or supplements of IF are briefly reviewed, including the journal-h index proposed by Braun, Gl?nzel, and Schubert. Ascribing higher weighting factors to citations of individual papers proportionally to IF is considered to be a misuse of useful numerical indices based on citations. At most, one could turn this argument on its head and one can find reasons to ascribe an inverse proportionality relative to IF for individual citations: if a paper is considered worthy to be cited even if it was published in a low-IF journal, that citation ought to be worth more than if the citation would have been from a higher-impact journal. A weight factor reflecting the prestige of the citing author(s) may also be considered.     

Impact of open access on citation of scholarly publications in the field of civil engineering

The development of science is accompanied by growth of scholarly publications, primarily in the form of articles in peer-reviewed journals. Scientific work is often evaluated through the number of scientific publications in international journals and their citations. This article discusses the impact of open access (OA) on the number of citations for an institution from the field of civil engineering. We analyzed articles, published in 2007 in 14 international journals with impact factor, which are included in the Journal Citation Reports subject category “Civil Engineering”. The influence of open access on the number of citations was analyzed. The aim of our research was to determine if open access articles from the field of civil engineering receive more citations than non-open access articles. Based on the value of impact factor and ranking in quartiles, we also looked at the influence of the rank of journals on the number of citations, separately for OA and Non OA articles, in databases Web of Science (WOS), Scopus and Google Scholar. For 2,026 studied articles we found out that 22 % of them were published as OA articles. They received 29 % of all citations in the observed period. We can conclude by the significance level 5 % or less that in the databases WOS and Scopus the articles from top ranked journals (first quartile) achieved more citations than Non OA articles. This argument can be confirmed for some other journals from second quartile as well, while for the journals ranked into the third quartile it can’t be confirmed. This could be confirmed only partly for journals from the second quartile, and would not be confirmed for journals ranked into the third quartile. This shows that open access is not a sufficient condition for citation, but increases the number of citations for articles published in journals with high impact.     

Is there a correlation between journal impact factor and researchers’ performance? A study comprising the fields of clinical nephrology and neurosciences

Quantifying the scientific performance of investigators has become an integral part of decision-making in research policy. The aim of the present study was to evaluate if there is a correlation between journal impact factor (IF) and researchers’ influence among a selected group of Brazilian investigators in the fields of clinical nephrology and neurosciences. This study was based on 94 senior investigators (36 in clinical nephrology and 58 in clinical neurosciences) receiving productivity scholarships from the Brazilian Council for Scientific and Technological Development (CNPq) according to a list provided by the agency in February 2009. Scientific performance indicators included in the analysis were: number of papers indexed by the Web of Science and Scopus databases, number of citations, h- and m-index. IFs were analyzed as (1) cumulative IF (∑IF), (2) IF adjusted by time (IF/t), and (3) average IF. There was a moderate positive correlation only between ∑IF and two indicators: total number of citations (P < 0.001) and h-index (P < 0.001). There was also a positive correlation between IF/t and m-index (P < 0.001). There was an agreement in these correlations between both groups (clinical nephrology and neurosciences). No significant correlation between the average IF and any of the scientific indicators was detected. A cut-off of 10.53 for IF/t showed the best performance in predicting researchers with m-index equal to or greater than 1. According to our findings, other qualitative and quantitative instruments rather than IF are clearly needed for identifying researchers with outstanding scientific output.     

Surviving bad times: The role of citations,self-citations and numbers of citable items in recovery of the journal impact factor after at least four years of continuous decreases

We studied the influence of the number of citations, the number of citable items and the number of journal self-citations on increases in the impact factor (IF) in 123 journals from the Journal Citation Reports database in which this scientometric indicator had decreased during the previous four years. In general, we did not find evidence that abuse of journal self-citations contributed to the increase in the impact factor after several years of decreases.     

More reprint requests, more citations?

Reprint requests are commonly used to obtain a copy of an article. This study aims to correlate the number of reprint requests from a 10-year-sample of articles with the number of citations. The database contained 28 articles published in over a 10-year-period (1992-2001). For each separate article the number of citations and and the number of reprint requests were retrieved. In total 303 reprint requests were analysed. Reviews (median 9, range 1 to 95) and original articles (median 8, range 1-36) attracted most reprint requests. There was an excellent correlation between the number of requests and citations to article (two-tailed non-parametric Spearman rank test r = 0.55; 95% confidence interval 0.18-0.78, P < 0.005). Articles that received most reprint requests are cited more often. This revised version was published online in August 2006 with corrections to the Cover Date.     

Effect on the journal impact factor of the number and document type of citing records: a wide-scale study

We studied the effect on journal impact factors (JIF) of citations from documents labeled as articles and reviews (usually peer reviewed) versus citations coming from other documents. In addition, we studied the effect on JIF of the number of citing records. This number is usually different from the number of citations. We selected a set of 700 journals indexed in the SCI section of JCR that receive a low number of citations. The reason for this choice is that in these instances some citations may have a greater impact on the JIF than in more highly-cited journals. After excluding some journals for different reasons, our sample consisted of 674 journals. We obtained data on citations that contributed to the JIF for the years 1998?C2006. In general, we found that most journals obtained citations that contribute to the impact factor from documents labeled as articles and reviews. In addition, in most of journals the ratio between citations that contributed to the impact factor and citing records was greater than 80% in all years. Thus, in general, we did not find evidence that citations that contributed to the impact factor were dependent on non-peer reviewed documents or only a few citing records.     

Journal status

Summary The status of an actor in a social context is commonly defined in terms of two factors: the total number of endorsements the actor receives from other actors and the prestige of the endorsing actors. These two factors indicate the distinction between popularity and expert appreciation of the actor, respectively. We refer to the former as popularity and to the latter as prestige. These notions of popularity and prestige also apply to the domain of scholarly assessment. The ISI Impact Factor (ISI IF) is defined as the mean number of citations a journal receives over a 2 year period. By merely counting the amount of citations and disregarding the prestige of the citing journals, the ISI IF is a metric of popularity, not of prestige. We demonstrate how a weighted version of the popular PageRank algorithm can be used to obtain a metric that reflects prestige. We contrast the rankings of journals according to their ISI IF and their Weighted PageRank, and we provide an analysis that reveals both significant overlaps and differences. Furthermore, we introduce the Y-factor which is a simple combination of both the ISI IF and the weighted PageRank, and find that the resulting journal rankings correspond well to a general understanding of journal status.     

The distribution of the uncitedness factor and its functional relation with the impact factor

The uncitedness factor of a journal is its fraction of uncited articles. Given a set of journals (e.g. in a field) we can determine the rank-order distribution of these uncitedness factors. Hereby we use the Central Limit Theorem which is valid for uncitedness factors since it are fractions, hence averages. A similar result was proved earlier for the impact factors of a set of journals. Here we combine the two rank-order distributions, hereby eliminating the rank, yielding the functional relation between the impact factor and the uncitedness factor. It is proved that the decreasing relation has an S-shape: first convex, then concave and that the inflection point is in the point (μ′, μ) where μ is the average of the impact factors and μ′ is the average of the uncitedness factors.     

Author self-citations in the field of ecology

Ecologists writing research articles frequently cite their own papers. Self-citations are frequent in science, but the reasons behind abnormally high rates of self-citations are questionable. My goals were to assess the prevalence of author self-citations and to identify the combination of attributes that best predict high levels of self-citations in ecology articles. I searched 643 articles from 9 different ecology journals of various impact factors for synchronous (i.e., within reference lists) and diachronous (i.e., following publication) self-citations, using the Web of Science online database. I assessed the effect of the number of authors, pages, and references/citations, the proportion of diachronous/synchronous self-citations, and the impact factor, on the proportion of synchronous and diachronous self-citations separately. I compared various candidate models made of these covariates using Akaike’s Information Criterion. On average, ecologists made 6.0 synchronous self-citations (12.8% of references), and 2.5 diachronous self-citations (25.5% of citations received 2.8 to 4.5 years after publication) per article. The best predictor of the proportion of synchronous self-citations was the number of authors. My study is the first to report recidivism in the inclusion of self-citations by researchers, i.e., the proportion of diachronous self-citations was best explained by the proportion of synchronous self-citations. The proportion of self-citations also increased with the number of pages and the impact factor of ecology journals, and decreased with the number of references/citations. Although a lot of variance remained unexplained, my study successfully showed regularities in the propensity of ecologists to include self-citations in their research articles.     

Measuring impact by a full option method and the notion of bibliometric spectra

A full option method for determining impact takes into account citations to all cited publications, instead of limiting the analysis to ISI-publications only, as usually done in the standard method. The method was tested for the 258 early Ghent professors, teaching in 6 different faculties. The impact of monographs is, in general, much larger than the impact of articles (whether of ISI-type or not). This result remains valid for all six faculties separately. Limiting the bibliometric visibility to ISI-publications reduces the number of citations to only 16%. Bibliometric spectra are presented, in which citations, cited publications and their impact are shown in function of the year of publication. The number of cited publications is always important to expose the influence of activity (production) upon bibliometric scores. For the faculty of Arts, the citations to early professors are compared with those obtained for the presentday generation: the bibliometric spectrum for the former group is rather discontinuous (showing a large erosion in the number of citations by year), whereas that of the latter is continuous. The Ghent citation data are also compared with those given internationally in the same period.     

The effect of document types and sizes on the scaling relationship between citations and co-authorship patterns in management journals

The aim of this paper is to explore the power-law relationship between citation-based performance (CBP) and co-authorship patterns for papers in management journals by analyzing its behavior according to the type of documents (articles and reviews) and the number of pages of documents. We analyzed 36,241 papers that received 239,172 citations. The scaling exponent of CBP for article papers was larger than for reviews. Citations to articles increased 2^1.67 or 3.18 times each time the number of article papers published in a year in management journals doubled. The citations to reviews increased 2^1.29 or 2.45 times each time the number of reviews published in a year in management journals doubled. The scaling exponent for the power-law relationship of citation-based performance according to number of pages of papers was 1.44 ± 0.05 for articles and 1.25 ± 0.05 for reviews. The citations to articles increased faster than citation to reviews. The scaling exponent for the power-law of citation-based performance to co-authored articles was higher than single-authored articles. For reviews the scaling exponent was the same for the relationship between citation based performance and the number of reviews. Citations increased faster in single authored reviews than co-authored reviews.     

A bibliometric analysis of hydrogen energy literature, 1965–2005

The present study explores the characteristics of hydrogen energy literature from 1965 to 2005 based on the database of Science Citation Index Expanded (SCIE) and its implication using the bibliometric techniques. The results of this work reveal that the literature on hydrogen energy grows exponentially with an annual growth rate of about 18% for the last decade. Most of document type is in the form of journal articles or meeting abstracts, constituting 90.17% of the total literature and English is the predominant language (94.66%). USA, Japan and China are the three biggest contributing countries on hydrogen energy literature publishing, 25.8%, 14.9%, 7.7%, respectively. The Chinese Academy of Sciences in China is the largest contributor publishing 308 papers. The journal literature on hydrogen energy does not confirm the typical S-shape for the Bradford-Zipf plot, but five core journals, i.e. International Journal of Hydrogen Energy, Journal of Power Source, Journal of the Electrochemical Society, Solid State Ionics, and Electrochimica Act, contributing about 41% can be identified. Journals with highly cited articles and most highly cited articles are also identified, in which the most highly cited article receives more than 1,000 citations.     

The effect of characteristics of title on citation rates of articles

This study aimed to assess the association between some features of articles title and number of citations in a volume of Addictive Behavior journal. All research articles published in the volume number 32 (2007) in the Addictive Behaviors journal (n = 302) were analyzed by two independent authors. For each article, the following information has been extracted: number of citations up to June 2013 in the Scopus citation database, type of and characteristics of titles, having different words in the keywords, reference to place and presence of an acronym. The summary statistics showed that mean number of citation was 16.36 ± 19.55 times. Articles with combinational title (use of a hyphen or a colon separating different ideas within a sentence) and articles with different words in the keywords (at least two different keywords) had higher number of citations. The number of citations was not correlated with the number of words in the title (r = 0.05, P = 0.325). Our results suggested that some features in the paper such as type of the title and articles with keywords different from words included in the title can help to predict the number of citation counts. These findings can be used by authors and reviewers in order to maximize the impact of articles. The length of title is not associated with citation counts. Therefore, the guide for authors of journals can be more flexible regarding the length of the title.     

Online determination of the journal impact factor and its international properties

The article describes the method for the online determination of the journal impact factor (JIF). The method is very simple and can be used both for the ISI defined journal impact factor and for the calculation of other generalised journal impact factors. But the direct online method fails for non-ISI journals i.e. journals not indexed by ISI to the three citation databases. For such journals only the “External Cited Impact Factor” associated with citations from ISI journals (ECIFisi) can be determined online by the common method. As an extra benefit the online method makes available the determination of the geographical distribution of citations and citable units in relation to any given JIF, i.e. the international impact for a particular journal in a given year. The method is illustrated by calculating the generalised JIF, self-citations and ECIF(isi) as well as the international impact for Journal of Documentation and Scientometrics.     

Evaluating journal performance metrics

Journals have been ranked on the basis of impact factors for a long time. This is a quality indicator, and often favours review journals with few articles. Integrated impact indicators try to factor in size (quantity) as well, and are correlated with total number of citations. The total number of papers in a portfolio can be considered a zeroth order performance indicator and the total number of citations a first order performance indicator. Indicators like the h-Index and the g-Index are actually performance indicators in that they integrate both quality and quantity assessment into a single number. The p-Index is another variant of this class of performance indicators and is based on the cubic root of a second order performance indicator called the exergy indicator. The Eigenfactor score and article influence are respectively first order quantity and quality indicators. In this paper, we confirm the above relationships.     

Are leaders really leading? Journals that are first in Web of Science subject categories in the context of their groups

This study focuses on journals that lead their Web of Science (WoS) subject category ranking when the usual 2-year window for the Journal Impact Factor (JIF2) is used as the ranking variable, and examines evidence that contradicts their top-ranked position in the context of their group. The source data were obtained from all 177 WoS subject categories in the Science Edition 2015 Journal Citation Reports (JCR). I compared journals in each WoS subject category with leaders in terms of JIF2, number of citable items and number of citations that contribute to the JIF2. Rankings were calculated with alternative metrics (for example, the Journal Impact Factor without self-citations and the eigenfactor), and the minimum reduction in the number of citations that would displace the top-ranked journal from its leading position was also calculated. In addition, the stability of rankings over time, the number of WoS subject categories in which journals are leaders, the publishers that own leading journals, and the percentages of research articles (as opposed to review articles) published in different journals were also studied. In general, leading journals are not necessarily the top-ranked in terms of citations received or the number of citable items they publish. In addition, most leaders maintained their position when other metrics were used instead the JIF2, although rankings based on the eigenfactor were at variance with this finding. The distribution of publishers was highly skewed, with a linear relationship between the cumulative number of publishers owning the top-ranked journal and the cumulative number of WoS subject categories. In only 85 subject categories (48%) the percentage of research articles (not reviews) in the number of citable items was greater than the mean percentage for the subject category. In 31 instances, leaders did not publish any research articles, but only reviews.     

Evolution of the publications in clinical neurology: scientific impact of different countries during the 2000–2009 period

We analyzed the productivity and visibility of publications on the subject category of Clinical Neurology by countries in the period 2000–2009. We used the Science Citation Index Expanded database of the ISI Web of Knowledge. The analysis was restricted to the citable documents. Bibliometric indicators included the number of publications, the number of citations, the median and interquartile range of the citations, and the h-index. We identified 170,483 publications (84.9 % original articles) with a relative increase of 28.5 % throughout the decade. Fourteen countries published over 2,000 documents in the decade and received more than 50,000 citations. The average of citations received per publication was 8 (interquartile range: 3–20) and the h-index was 261. USA was the country with the highest number of publications, followed by Germany, Japan, the UK and Italy. Moreover, USA publications had the largest number of citations received (44.5 % of total), followed by the UK, Germany, Canada, and Italy. On the other hand, Sweden, the Netherlands and the UK had the highest median citations for their total publications. During the period 2000–2009 there was a significant increase in Clinical Neurology publications. Most of the publications and citations comprised 14 countries, with the USA in the first position. Interestingly, most of the publications and citations originated from only 14 countries, with European countries with relatively low population, such as Switzerland, Austria, Sweden, Belgium, and the Netherlands, in this top group.     

An informetric model for the Hirsch-index

Summary The h-index (or Hirsch-index) was defined by Hirsch in 2005 as the number h such that, for a general group of papers, h papers received at least h citations while the other papers received no more than h citations. This definition is extended here to the general framework of Information Production Processes (IPPs), using a source-item terminology. It is further shown that in each practical situation an IPP always has a unique h-index. In Lotkaian systems h = T^1/a, where T is the total number of sources and α is the Lotka exponent. The relation between h and the total number of items is highlighted.     

Brazil’s growing production of scientific articles—how are we doing with review articles and other qualitative indicators?

This article identifies scientific fields in Brazil that have been generating new knowledge, their evolution, tendencies and the relationship between scientific production and the National Postgraduate Program (Programa Nacional de Pós-Graduação—PNPG). It works with review articles and assesses: (a) articles published as reviews by international databases; (b) the growth of Brazilian participation in that context; (c) institutional participation; (d) the predominant fields of knowledge; (e) the most productive authors, and (f) periodicals that published the greatest number of review articles by Brazilian authors. The 5,348 review articles published between 2000 and 2009 were made available in 1,309 scientific publications, and the fields that published the most reviews were Pharmacology, Chemistry, Neurosciences, Biochemistry and Molecular Biology, Psychiatry, Neurology, Endocrinology and Internal Medicine. The reviews were produced by 27,096 authors under the auspices of 20 institutions which, together, answer for 95 % of the Brazilian production, and are public, excepting for the Pontifícia Universidade Católica do Rio Grande do Sul (PUC-RS). In the international scenario, we find that 31 countries are responsible for 90.11 % of the total scientific production and 94.08 % of the review articles. To establish a comparison between Brazil and its closest competitors, these countries can be classified arbitrarily in three groups: (1) countries with a large number of review articles (>3100), an average number of citations above 18, and an h-index greater than 95; (2) countries with a significant production of articles (between 2,000 and 3,000), average number of citations between 12 and 17, and an h-index below 95. Brazil is in the third group, (3) formed by countries with a lower level of production and the two qualitative indicators at opposite poles: the average of citations on a par with the first group (>18) and h-indexes like those of the second group (<95).