Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles

Context. The use of citation frequency and impact factor as measures of research quality and journal prestige is being criticized. Citation frequency is augmented by self-citation and for most journals the majority of citations originate from a minority of papers. We hypothesized that citation frequency is also associated with the geographical origin of the research publication. Objective. We determined whether citations originate more frequently from institutes that are located in the same country as the authors of the cited publication than would be expected by chance. Design. We screened citations referring to 1200 cardiovascular publications in the 7 years following their publication. For the 1200 citation recipient publications we documented the country where the research originated (9 countries/regions) and the total number of received citations. For a selection of 8864 citation donor papers we registered the country/region where the citing paper originated. Results. Self-citation was common in cardiovascular journals (n = 1534, 17.8%). After exclusion of self-citation, however, the number of citations that originated from the same country as the author of the citation recipient was found to be on average 31.6% higher than would be expected by chance (p<0.01 for all countries/regions). In absolute numbers, nation oriented citation bias was most pronounced in the USA, the country with the largest research output (p<0.001). Conclusion. Citation frequency was significantly augmented by nation oriented citation bias. This nation oriented citation behaviour seems to mainly influence the cumulative citation number for papers originating from the countries with a larger research output.     

A macro study of self-citation

This study investigates the role of self-citation in the scientific production of Norway (1981-1996). More than 45,000 publications have been analysed. Using a three-year citation window we find that 36% of all citations represent author self-citations. However, this percentage is decreasing when citations are traced for longer periods. We find the highest share of self-citation among the least cited papers. There is a strong positive correlation between the number of self-citations and the number of authors of the publications. Still, only a minor part of the overall increase in citation rates that can be found for multi-authored papers is due to self-citations. Also, the share of self-citation shows significant variations among different scientific disciplines. The results are relevant for the discussion concerning use of citation indicators in research assessments. This revised version was published online in August 2006 with corrections to the Cover Date.     

Author self-citations in the field of ecology

Ecologists writing research articles frequently cite their own papers. Self-citations are frequent in science, but the reasons behind abnormally high rates of self-citations are questionable. My goals were to assess the prevalence of author self-citations and to identify the combination of attributes that best predict high levels of self-citations in ecology articles. I searched 643 articles from 9 different ecology journals of various impact factors for synchronous (i.e., within reference lists) and diachronous (i.e., following publication) self-citations, using the Web of Science online database. I assessed the effect of the number of authors, pages, and references/citations, the proportion of diachronous/synchronous self-citations, and the impact factor, on the proportion of synchronous and diachronous self-citations separately. I compared various candidate models made of these covariates using Akaike’s Information Criterion. On average, ecologists made 6.0 synchronous self-citations (12.8% of references), and 2.5 diachronous self-citations (25.5% of citations received 2.8 to 4.5 years after publication) per article. The best predictor of the proportion of synchronous self-citations was the number of authors. My study is the first to report recidivism in the inclusion of self-citations by researchers, i.e., the proportion of diachronous self-citations was best explained by the proportion of synchronous self-citations. The proportion of self-citations also increased with the number of pages and the impact factor of ecology journals, and decreased with the number of references/citations. Although a lot of variance remained unexplained, my study successfully showed regularities in the propensity of ecologists to include self-citations in their research articles.     

Is citation analysis a legitimate evaluation tool?

A comprehensive discussion on the use of citation analysis to rate scientific performance and the controversy surrounding it. The general adverse criticism that citation counts include an excessive number of negative citations (citations to incorrect results worthy of attack), self-citations (citations to the works of the citing authors), and citations to methodological papers is analyzed. Included are a discussion of measurement problems such as counting citations for multiauthored papers, distinguishing between more than one person with the same last name (homographs), and what it is that citation analysis actually measures. It is concluded that as the scientific enterprise becomes larger and more complex, and its role in society more critical, it will become more difficult, expensive and necessary to evaluate and identify the largest contributors. When properly used, citation analysis can introduce a useful measure of objectivity into the evaluation process at relatively low financial cost.Modification of a chapter in E. Garfield:Citation Indexing: Its Theory and Application in Science, Technology and the Humanities, New York, Wiley, 1979.     

Detecting <Emphasis Type="Italic">h</Emphasis>-index manipulation through self-citation analysis

The h-index has received an enormous attention for being an indicator that measures the quality of researchers and organizations. We investigate to what degree authors can inflate their h-index through strategic self-citations with the help of a simulation. We extended Burrell’s publication model with a procedure for placing self-citations, following three different strategies: random self-citation, recent self-citations and h-manipulating self-citations. The results show that authors can considerably inflate their h-index through self-citations. We propose the q-index as an indicator for how strategically an author has placed self-citations, and which serves as a tool to detect possible manipulation of the h-index. The results also show that the best strategy for an high h-index is publishing papers that are highly cited by others. The productivity has also a positive effect on the h-index.     

A scientometric analysis of international LIS journals: Productivity and characteristics

This paper presents a quantitative study of productivity, characteristics and various aspects of global publication in the field of library and information science (LIS). A total of 894 contributions published in 56 LIS journals indexed in SSCI during the years of 2000-2004 were analyzed. A total of 1361 authors had contributed publications during the five years. The overwhelming majority (89.93%) of them wrote one paper. The average number of authors per paper is 1.52. All the studied papers were published in English. The sum of research output of the authors form USA and UK reaches 70% of the total productivity. Most papers received few citations. Each article received on an average 1.6 citations and the LIS researchers cite mostly latest articles. About 48% of citing authors had tendency of self-citation. The productive authors, their contribution and authorship position are listed to indicate their productivity and degree of involvement in their research publications.     

Citation pattern and lifespan: a comparison of discipline,institution, and individual

Citation frequency is often used in hiring and tenure decisions as an indicator of the quality of a researcher’s publications. In this paper, we examine the influence of discipline, institution, journal impact factor, length of article, number of authors, seniority of author, and gender on citation rate of top-cited papers for academic faculty in geography and forestry departments. Self-citation practices and patterns of citation frequency across post-publication lifespan were also examined. Citation rates of the most-highly cited paper for all tenured forestry (N = 122) and geography (N = 91) faculty at Auburn University, Michigan State University, Northern Arizona University, Oklahoma State University, Pennsylvania State University, Texas A&M University, University of Florida, University of Massachusetts, University of Washington, and Virginia Tech were compared. Foresters received significantly more citations than geographers (t = 2.46, P = 0.02) and more senior authors received more citations than junior researchers (r ² = 0.14, P = 0.03). Articles published in journals with higher impact factors also received more citations (r ² = 0.28, P = 0.00). The median self-citation rate was 10% and there was no temporal pattern to the frequency of citations received by an individual article (x ² = 176). Our results stress the importance of only comparing citation rates within a given discipline and confirm the importance of author-seniority and journal rankings as factors that influence citation rate of a given article.     

Self-citations at the meso and individual levels: effects of different calculation methods

This paper focuses on the study of self-citations at the meso and micro (individual) levels, on the basis of an analysis of the production (1994–2004) of individual researchers working at the Spanish CSIC in the areas of Biology and Biomedicine and Material Sciences. Two different types of self-citations are described: author self-citations (citations received from the author him/herself) and co-author self-citations (citations received from the researchers’ co-authors but without his/her participation). Self-citations do not play a decisive role in the high citation scores of documents either at the individual or at the meso level, which are mainly due to external citations. At micro-level, the percentage of self-citations does not change by professional rank or age, but differences in the relative weight of author and co-author self-citations have been found. The percentage of co-author self-citations tends to decrease with age and professional rank while the percentage of author self-citations shows the opposite trend. Suppressing author self-citations from citation counts to prevent overblown self-citation practices may result in a higher reduction of citation numbers of old scientists and, particularly, of those in the highest categories. Author and co-author self-citations provide valuable information on the scientific communication process, but external citations are the most relevant for evaluative purposes. As a final recommendation, studies considering self-citations at the individual level should make clear whether author or total self-citations are used as these can affect researchers differently.     

Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers

This paper focuses the attention on the ch-index, a recent bibliometric indicator similar to the Hirsch (h) index, to evaluate the published research output of a scientist (Ajiferuke and Wolfram, Proceedings of the 12th international conference of the international society for scientometrics and informetrics. Rio de Janeiro, pp. 798–808, 2009). Ch-index is defined as the number such that, for a general group of scientific publications, ch publications are cited by at least ch different citers while the other publications are cited by no more than ch different citers. The basic difference from the classical h is that, according to ch, the diffusion of one author’s publication is evaluated on the basis of the number of different citing authors (or citers), rather than the number of received citations. The goal of this work is to discuss the pros and cons of ch and identify its connection with h. A large sample of scientists in the Quality Engineering/Management field are analyzed so as to investigate the novel indicator’s characteristics. Then, the analysis is preliminarily extended to other scientific disciplines. The most important result is that ch is almost insensitive to self-citations and/or citations made by recurrent citers, and it can be profitably used for complementing h.     

Exploring the analytical potential of comparing citing and cited source items

Summary Comparing properties of citing and cited source items opens a wide variety of analytical possibilities. In a study of citations among papers in the journal Scientometrics a number of analytical themes are identified. The analysis shows: the way in which a citation graph can be decomposed into different subparts; country specific citation patterns; the effects of self-citations and domestic citations; the mapping of cited author relationships using direct citation and co-citation links; and time slicing effects on impact ranking of countries and papers.     

Sustained impact of publications of C.V. Raman

C.V. Raman is being acknowledged by worldwide physics community for his classic works. The present study has made an effort to analyze how much impact in number of citation receiving for his publications. Of course, there was a lack of tools for such a study some years back. The study has limited to the database Science Citation Index for the period 1982–2005. The noteworthy results are: One third of his research papers have been cited at least once; The research papers published during 1918–1940 could make remarkable impact; Three of his papers have shown an upward growth in number of citations receiving; The total citations to papers of age 46 and 54 as on the year 1982 accounted for more than 50 per cent of the total citations received; Research works in the ‘Acoustics’ area have been cited more than any other area of his works; Eponymal citations are to be explored and analysed to understand the real impact of his works.     

A quasi-quantitative citation model

On the basis of investigating author's opinion on citing motivations of chemistry papers aquasi-quantitative model for citing is suggested. The model selects professional and nonprofessional motivations of citing and introduces thecitation threshold concept which tries to characterize the effect of citing motivations quantitatively. Possible reasons for missing citations are also treated. Mean ages of real and of self-citations were calculated by subtracting the average of the publication years of cited papers from the publication year of the citing publication. The difference between the mean ages may characterize thesynchronity of the author's research in comparison with those working on similar topics. The paper introduces thecitation strategy indicator which relates impact factors of cited periodicals with the mean impact factor of periodicals in the corresponding research subfield.     

Citation analysis of research performer quality

Background: Citation analysis for evaluative purposes typically requires normalization against some control group of similar papers. Selection of this control group is an open question. Objectives: Gain a better understanding of control group requirements for credible normalization. Approach: Performed citation analysis on prior publications of two proposing research units, to help estimate team research quality. Compared citations of each unit"s publications to citations received by thematically and temporally similar papers. Results: Identification of thematically similar papers was very complex and labor intensive, even with relatively few control papers selected. Conclusions: A credible citation analysis for determining performer or team quality should have the following components: – Multiple technical experts to average out individual bias and subjectivity; – A process for comparing performer or team output papers with a normalization base of similar papers; – A process for retrieving a substantial fraction of candidate normalization base papers; Manual evaluation of many candidate normalization base papers to obtain high thematic similarity and statistical representation.     

Does self-citation pay?

Self-citations — those where authors cite their own works — account for a significant portion of all citations. These self-references may result from the cumulative nature of individual research, the need for personal gratification, or the value of self-citation as a rhetorical and tactical tool in the struggle for visibility and scientific authority. In this article we examine the incentives that underlie self-citation by studying how authors’ references to their own works affect the citations they receive from others. We report the results of a macro study of more than half a million citations to articles by Norwegian scientists that appeared in the Science Citation Index. We show that the more one cites oneself the more one is cited by other scholars. Controlling for numerous sources of variation in cumulative citations from others, our models suggest that each additional self-citation increases the number of citations from others by about one after one year, and by about three after five years. Moreover, there is no significant penalty for the most frequent self-citers — the effect of self-citation remains positive even for very high rates of self-citation. These results carry important policy implications for the use of citations to evaluate performance and distribute resources in science and they represent new information on the role and impact of self-citations in scientific communication.     

Research performance indicators for university departments: A study of an agricultural university

The present bibliometric study extends previous work by focusing on the research performance of departments in the natural and life sciences, the social and behavioral sciences, and the humanities. The present study covers all 70 departments from one agricultural university, and several veterinary departments of a second university. The impact analysis was extended by including other types of documents than journal articles. For about a third of the departments, publications not covered in citation indexes accounted for at least 30% of the citations to their total oeuvre. To deal with different citation and publication habits in the various fields, both short-term and medium-term impact assessments were made. The commonly used three year window is not universally applicable, as our results show. The inclusion of self-citations forms an important source of error in the ratio of actual/expected impact. To cope with this, the trend and level of self-citations was compared at university level with that in a matched sample of publications. Moreover, at a departmental level, self-citation rates were used to detect departments with divergent levels of self-citation. The expected impact of journals accounted for only 18% of the variance in actual impact. Comparison of bibliometric indicators with two peer evaluations showed that the bibliometric impact analyses provided important additional information.     

International earth science literature from Turkey — 1970–2005: Trends and possible causes

We investigated the publication trends in the international earth science literature coming out of Turkey in the period of 1970–2005 using the Science Citation Index Expanded database. A database of 2310 earth science publications with at least one of the authors with an address in Turkey was compiled. The number of earth science publications from Turkey shows a very rapid increase starting in the 1990’s in parallel with the increase in the total scientific output of Turkey. In the last decade the annual growth rate has been 16%. There was also a concomitant increase in the number of citations. The causes of the sharp increase in the publication numbers are, in order of importance, changes in the rules of academic promotion and appointment, changes in academic attitudes towards publishing, increasing support for research, financial incentives for publishing, and expansion of higher education. However, the sharp increase in the publication numbers was not accompanied by a similar increase in the impact of the publications as measured by the citations. Although publications with first authors from outside Turkey make up only 20% of the Turkish earth science publications in the period 1970–2005, these account for 38% of the total citations, and constitute 48 out of 100 most cited papers.     

Some bibliometric correlates of quality in scientific research

The following kinds of data were collected on three samples of cancer research literature representing three levels of quality: (1) collaboration as measured by the number of authors per paper, (2) quantitative productivity of countries, (3) diachronous citations covering the first five years of publication, (4) total self-citations, (5) proportions of self-citations made by first-named authors, and (6) the extent of dispersion of articles among journals. Analyses showed that as the number of authors per paper increases, the proportion of high quality papers also increases and the Collaborative Index can be used to measure quality in the aggregate. It was found that the quantity and quality of cancer research done in a country are positively related. All analyses of the citation data confirmed the hypotheses that highly rated papers are significantly more highly cited than average papers and the rates of uncitedness decline with quality. The proportion of self-citations to total citations decreases with increasing quality and, on average, first-named authors of quality papers cite them proportionally fewer times than first-named authors of run-of-the mill papers do. This study also shows that, as quality increases, the extent of literature scatter or dispersion increases.     

Analysis of publication quality in a Cancer Research Institute

The paper presents an experimental method for the evaluation of scientific papers in the field of oncology and related disciplines developed at the National Institute for Cancer Research (IST), Genoa, Italy. The method is based on the partitioning of categories of theScience Citation Index-Journal Citation Reports (SCI-JCR) into deciles, thus normalizing Impact Factor (IF), in order to guage the quality of the productivity. A second parameter related to the number of staff of each department co-authoring a given paper has been introduced for the allocation of Institute funding. The following studies have been carried to compare the assigned score and the average number of citations of papers published by a research group. The identification of correctives is in progress. The method provides a basis for a possible method to judge the quality of publications from within a research organization, and should be reproducible independently of the disciplines considered.     

CAD: an algorithm for citation-anchors detection in research papers

Citations are very important parameters and are used to take many important decisions like ranking of researchers, institutions, countries, and to measure the relationship between research papers. All of these require accurate counting of citations and their occurrence (in-text citation counts) within the citing papers. Citation anchors refer to the citation made within the full text of the citing paper for example: ‘[1]’, ‘(Afzal et al, 2015)’, ‘[Afzal, 2015]’ etc. Identification of citation-anchors from the plain-text is a very challenging task due to the various styles and formats of citations. Recently, Shahid et al. highlighted some of the problems such as commonality in content, wrong allotment, mathematical ambiguities, and string variations etc in automatically identifying the in-text citation frequencies. The paper proposes an algorithm, CAD, for identification of citation-anchors and its in-text citation frequency based on different rules. For a comprehensive analysis, the dataset of research papers is prepared: on both Journal of Universal Computer Science (J.UCS) and (2) CiteSeer digital libraries. In experimental study, we conducted two experiments. In the first experiment, the proposed approach is compared with state-of-the-art technique over both datasets. The J.UCS dataset consists of 1200 research papers with 16,000 citation strings or references while the CiteSeer dataset consists of 52 research papers with 1850 references. The total dataset size becomes 1252 citing documents and 17,850 references. The experiments showed that CAD algorithm improved F-score by 44% and 37% respectively on both J.UCS and CiteSeer dataset over the contemporary technique (Shahid et al. in Int J Arab Inf Technol 12:481–488, 2014). The average score is 41% on both datasets. In the second experiment, the proposed approach is further analyzed against the existing state-of-the-art tools: CERMINE and GROBID. According to our results, the proposed approach is best performing with F1 of 0.99, followed by GROBID (F1 0.89) and CERMINE (F1 0.82).