Comparison of citation and usage indicators: the case of oncology journals

It is the objective of this article to examine in which aspects journal usage data differ from citation data. This comparison is conducted both at journal level and on a paper by paper basis. At journal level, we define a so-called usage impact factor and a usage half-life in analogy to the corresponding Thomson’s citation indicators. The usage data were provided from Science Direct, subject category “oncology”. Citation indicators were obtained from JCR, article citations were retrieved from SCI and Scopus. Our study shows that downloads and citations have different obsolescence patterns. While the average cited half-life was 5.6 years, we computed a mean usage half-life of 1.7 years for the year 2006. We identified a strong correlation between the citation frequencies and the number of downloads for our journal sample. The relationship was lower when performing the analysis on a paper by paper basis because of existing variances in the citation-download-ratio among articles. Also the correlation between the usage impact factor and Thomson’s journal impact factor was “only” moderate because of different obsolescence patterns between downloads and citations.     

Bias in the journal impact factor

The ISI journal impact factor (JIF) is based on a sample that may represent half the whole-of-life citations to some journals, but a small fraction (<10%) of the citations accruing to other journals. This disproportionate sampling means that the JIF provides a misleading indication of the true impact of journals, biased in favour of journals that have a rapid rather than a prolonged impact. Many journals exhibit a consistent pattern of citation accrual from year to year, so it may be possible to adjust the JIF to provide a more reliable indication of a journal’s impact.     

Correlation between the Journal Impact Factor and three other journal citation indices

To determine the degree of correlation among journal citation indices that reflect the average number of citations per article, the most recent journal ratings were downloaded from the websites publishing four journal citation indices: the Institute of Scientific Information’s journal impact factor index, Eigenfactor’s article influence index, SCImago’s journal rank index and Scopus’ trend line index. Correlations were determined for each pair of indices, using ratings from all journals that could be identified as having been rated on both indices. Correlations between the six possible pairings of the four indices were tested with Spearman’s rho. Within each of the six possible pairings, the prevalence of identifiable errors was examined in a random selection of 10 journals and among the 10 most discordantly ranked journals on the two indices. The number of journals that could be matched within each pair of indices ranged from 1,857 to 6,508. Paired ratings for all journals showed strong to very strong correlations, with Spearman’s rho values ranging from 0.61 to 0.89, all p < 0.001. Identifiable errors were more common among scores for journals that had very discordant ranks on a pair of indices. These four journal citation indices were significantly correlated, providing evidence of convergent validity (i.e. they reflect the same underlying construct of average citability per article in a journal). Discordance in the ranking of a journal on two indices was in some cases due to an error in one index.     

Parameter identification of the observed citation distribution

Based on the transfer function model of the observed citation distribution and the expression of the cumulative citation probability distribution, parameters of 12 citation distributions are identified from statistical data of age distributions of references of 10 journals in JCR using the parameter optimization fitting method. At same time, based on the steady state solution of differential equations of the publication delay process and data of publication delays of 10 journals, the publication delay parameters of every journal are identified using the fitting method. Identified parameters of every journal citation distribution are compared with the journal’s publication delay parameters and some valuable conclusions are deduced.     

Online determination of the journal impact factor and its international properties

The article describes the method for the online determination of the journal impact factor (JIF). The method is very simple and can be used both for the ISI defined journal impact factor and for the calculation of other generalised journal impact factors. But the direct online method fails for non-ISI journals i.e. journals not indexed by ISI to the three citation databases. For such journals only the “External Cited Impact Factor” associated with citations from ISI journals (ECIFisi) can be determined online by the common method. As an extra benefit the online method makes available the determination of the geographical distribution of citations and citable units in relation to any given JIF, i.e. the international impact for a particular journal in a given year. The method is illustrated by calculating the generalised JIF, self-citations and ECIF(isi) as well as the international impact for Journal of Documentation and Scientometrics.     

References,authors, journals and scientific disciplines underlying the sustainable development literature: a citation analysis

This paper identifies the main references, authors and journals influencing the sustainable development literature. The task is accomplished by means of a citation analysis based on the records of ISI Web of Science. We found that the core of sustainability thinking is framed by a pattern of landmark studies published around every 5 years. Only 380 publications have been cited at least ten times. References with the highest influence are those with a global dimension and large diffusion, such as Brundtland Commission’s “Our common future” (1987) and classics such as Meadows’ et al. “Limits to growth” (1972). The list of the most influential references over the period 1960–2005 is dominated by contributions from economics (particularly ecological economics) and environmental science, but includes many other disciplines such as urban planning, political sciences and sociology. References are also made to policy documents such as “Agenda 21”, one of the main outcomes of the Rio Summit in 1992. In analyzing citation trends, we found that classics, because of their high rates of citations per year, seem to have a more enduring and stable influence.     

The Journal’s Integrated Impact Index: a new indicator for journal evaluation

Journal impact factor (JIF) has been used for journal evaluation over a long time, but also accompanied by the continuing controversy. In this study, a new indicator, the Journal’s Integrated Impact Index (JIII) has been proposed for journal evaluation. In the JIII, one journal’s average citations per paper, total citations, and all journals’ average level of average citations per paper and total citations have been used to characterize the integrated impact of journals. Some contrastive analyses were carried out between JIII and JIF. The results show some interesting properties of the new indicator, and also reveal some relevant relationships among JIII, JIF, and other bibliometric indicators.     

Journal self-citations that contribute to the impact factor: Documents labeled “editorial material” in journals covered by the Science Citation Index

Summary We investigated the distribution of citations included in documents labeled by the ISI as “editorial material” and how they contribute to the impact factor of journals in which the citing items were published. We studied all documents classified by the ISI as “editorial material” in the Science Citation Index between 1999 and 2004 (277,231 records corresponding to editorial material published in 6141 journals). The results show that most journals published only a few documents that included 1 or 2 citations that contributed to the impact factor, although a few journals published many such documents. The data suggest that manipulation of the impact factor by publishing large amounts of editorial material with many citations to the journal itself is not a widely used strategy to increase the impact factor.     

Journal self-citation rates in ecological sciences

Impact factors are a widely accepted means for the assessment of journal quality. However, journal editors have possibilities to influence the impact factor of their journals, for example, by requesting authors to cite additional papers published recently in that journal thus increasing the self-citation rate. I calculated self-citation rates of journals ranked in the Journal Citation Reports of ISI in the subject category “Ecology” (n = 107). On average, self citation was responsible for 16.2 ± 1.3% (mean ± SE) of the impact factor in 2004. The self-citation rates decrease with increasing journal impact, but even high impact journals show large variation. Six journals suspected to request for additional citations showed high self-citation rates, which increased over the last seven years. To avoid further deliberate increases in self-citation rates, I suggest to take journal-specific self-citation rates into account for journal rankings.     

Structure of the impact factor of academic journals in the field of Education and Educational Psychology: Citations from editorial board members

Summary We present a new approach to study the structure of the impact factor of academic journals. This new method is based on calculation of the fraction of citations that contribute to the impact factor of a given journal that come from citing documents in which at least one of the authors is a member of the cited journal's editorial board. We studied the structure of three annual impact factors of 54 journals included in the groups “Education and Educational Research” and “Psychology, Educational” of the Social Sciences Citation Index. The percentage of citations from papers authored by editorial board members ranged from 0% to 61%. In 12 journals, for at least one of the years analysed, 50% or more of the citations that contributed to the impact factor were from documents published in the journal itself. Given that editorial board members are considered to be among the most prestigious scientists, we suggest that citations from papers authored by editorial board members should be given particular consideration.     

The journal download immediacy index (DII): experiences using a Chinese full-text database

Relationships between the journal download immediacy index (DII) and some citation indicators are studied. The Chinese full-text database CNKI is used for data collection. Results suggest that the DII can be considered as an independent indicator, but that it also has predictive value for other indicators, such as a journal’s h-index. In case a journal cannot yet have an impact factor—because its citation history within the database is too short—the DII can be used for a preliminary evaluation. The article provides results related to the CNKI database as a whole and additionally, some detailed information about agricultural and forestry journals.     

A quick and easy method to estimate the random effect on citation measures

A quick and easy method is presented to estimate the random fluctuations exhibited by citation measures. Applying this method allows for instance a better view on the ranking of journals (their so called pecking order), when the ranks are based on the number of recieved citations or on the impact factor of the journal.     

Galton 2011 revisited: a bibliometric journey in the footprints of a universal genius

Commemorating the 100th death anniversary of Francis Galton, this paper is a bibliometric impact analysis of the works of this outstanding scientist and predecessor of scientometrics. Citation analysis was done in Web of Science, Scopus and Google Scholar (Publish or Perish) in order to retrieve the most cited books and journal articles. Additionally references were identified where Galton was rather mentioned than cited in order to analyze the phenomenon of obliteration by incorporation. Finally occurrence counts of Galton’s works in obituaries, Festschrift, the website Galton.org, major encyclopaedias and biographical indexes were compared to citation counts. As an outcome Galton’s works are increasingly cited or mentioned. Obliteration (use of eponyms) applies to one-third of Galton’s works and seems to be typical for fields like mathematics or statistics, whereas citations are more common in psychology. The most cited books and journal articles are also the most mentioned with remarkable correlation. Overall citation analysis and occurrence counting are complementary useful methods for the impact analysis of the works of “giants”.     

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

     

Revisiting citation aging: a model for citation distribution and life-cycle prediction

The study of citation distribution provides retrospective and prospective picture of the evolving impact of a corpus of publications on knowledge community. All distribution models agree on the rise of the number of citations in the first years following the publication to reach a peak and then tend to be less cited when time passes. However, questions such as how long it will continue being cited and what is objectively the rate of the decline remain unanswered. Built up of simple polynomial function, the proposed model is proven to be suitable to represent the observed citation distribution over time and to interestingly identify with accuracy when the major loss of citations happens. I calculate from the model the ‘residual citations’ representing the citations kept after a long time period after publication year. I demonstrate that the residual citations may be greater than or equal to zero, meaning that the ‘life-cycle’ of the corpus is infinite, contrary to what some researches termed to be around 21 years. This model fits the observed data from SCI according to R-sq which is greater than 98.9%. Rather, it is very simple and easy to implement and can be used by not highly-skilled scientometric users. Finally, the model serves as a citation predictive tool for a corpus by determining the citations that would obtain at any time of its life-cycle.     

What makes a journal international? A case study using conservation biology journals

The qualitative label ‘international journal’ is used widely, including in national research quality assessments. We determined the practicability of analysing internationality quantitatively using 39 conservation biology journals, providing a single numeric index (IIJ) based on 10 variables covering the countries represented in the journals’ editorial boards, authors and authors citing the journals’ papers. A numerical taxonomic analysis refined the interpretation, revealing six categories of journals reflecting distinct international emphases not apparent from simple inspection of the IIJs alone. Categories correlated significantly with journals’ citation impact (measured by the Hirsch index), with their rankings under the Australian Commonwealth’s ‘Excellence in Research for Australia’ and with some countries of publication, but not with listing by ISI Web of Science. The assessments do not reflect on quality, but may aid editors planning distinctive journal profiles, or authors seeking appropriate outlets.     

Great expectations: The role of Open Access in improving countries’ recognition

Open Access movement has been proven to be capable to enhance the recognition of scientific outputs by improving their visibility. However, it is not clear how different entities benefit from the Open Access advantage; because, the recognition process is dominated by some psychological or realistic biases, resulting in an unequal distribution of citations between different entities. The biases may be exacerbated in Open Access world, e.g. due to the scientists uncertainty about the quality of Open Access materials, or quantitatively or qualitatively unequal presence of countries. Consequently, although, Open Access is able to achieve their potential citations, it is not unlikely that it increases the inequalities, depriving the already “have-nots”. Trying to illuminate how countries are benefiting from Open Access advantage, this study compares citation performances of the world’s countries in two journal sets, i.e. Open Access and non Open Access journals. The results of the analyses conducted at subject field level show that overall recognition gap between developed and less-developed blocks is widened by publishing in Open Access journals. The verification of individual countries’ performances confirms the finding by revealing that open-access-advantaged nations are mainly consisted of developed ones. However, some open-access-advantaged instances are from the less-developed block, which may promisingly suggest early heralds of Open Access potentialities to achieve the recognition of “lost sciences”, leading to relative reparation of the gap in future.     

The citation impact outside references — formal versus informal citations

In this study the amount of “informal” citations (i.e. those mentioning only author names or their initials instead of the complete references) in comparison to the “formal” (full reference based) citations is analyzed using some pioneers of chemistry and physics as examples. The data reveal that the formal citations often measure only a small fraction of the overall impact of seminal publications. Furthermore, informal citations are mainly given instead of (and not in addition to) formal citations. As a major consequence, the overall impact of pioneering articles and researchers cannot be entirely determined by merely counting the full reference based citations.     

Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation

The increasing use of bibliometric indicators in science policy calls for a reassessment of their robustness and limits. The perimeter of journal inclusion within ISI databases will determine variations in the classic bibliometric indicators used for international comparison, such as world shares of publications or relative impacts. We show in this article that when this perimeter is adjusted using a natural criterion for inclusion of journals, the journal impact, the variation of the most common country indicators (publication and citation shares; relative impacts) with the perimeter chosen depends on two phenomena. The first one is a bibliometric regularity rooted in the main features of competition in the open space of science, that can be modeled by bibliometric laws, the parameters of which are “coverage-independent” indicators. But this regularity is obscured for many countries by a second phenomenon, the presence of a sub-population of journals that does not reflect the same international openness, the nationally-oriented journals. As a result indicators based on standard SCI or SCISearch perimeters are jeopardized to a certain extent by this sub-population which creates large irregularities. These irregularities often lead to an over-estimation of share and an under-estimation of the impact, for countries with national editorial tradition, while the impact of a few mainstream countries arguably benefits from the presence of this sub-population. This revised version was published online in August 2006 with corrections to the Cover Date.