Characteristics of journal impact factors: The effects of uncitedness and citation distribution on the understanding of journal impact factors

Summary In this study, journal impact factors play a central role. In addition to this important bibliometric indicator, which evolves around the average impact of a journal in a two-year timeframe, related aspects of journal impact measurement are studied. Aspects like the output volume, the percentage of publications not cited, and the citation frequency distribution within a set timeframe are researched, and put in perspective with the 'classical' journal Impact Factor. In this study it is shown that these aspects of journal impact measurement play a significant role, and are strongly inter-related. Especially the separation between journals on the basis of the differences in output volume seems to be relevant, as can be concluded from the different results in the analysis of journal impact factors, the degree of uncitedness, and the share of a journal its contents above or below the impact factor value.     

Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window

In this article I study characteristics of the journal impact factor (JIF) computed using a 5-year citation window as compared with the classical JIF computed using a 2-year citation window. Since 2007 ISI-Thomson Reuters has published the new 5-year impact factor in the JCR database. I studied changes in the distribution of JIFs when the citation window was enlarged. The distributions of journals according their 5-year JIFs were very similar all years studied, and were also similar to the distribution according to the 2-year JIFs. In about 72% of journals, the JIF increased when the longer citation window was used. Plots of 5-year JIFs against rank closely followed a beta function with two exponents. Thus, the 5-year JIF seems to behave very similarly to the 2-year JIF. The results also suggest that gains in JIF with the longer citation window tend to distribute similarly in all years. Changes in these gains also tend to distribute similarly from 1 year to the following year.     

Correlation between the Journal Impact Factor and three other journal citation indices

To determine the degree of correlation among journal citation indices that reflect the average number of citations per article, the most recent journal ratings were downloaded from the websites publishing four journal citation indices: the Institute of Scientific Information’s journal impact factor index, Eigenfactor’s article influence index, SCImago’s journal rank index and Scopus’ trend line index. Correlations were determined for each pair of indices, using ratings from all journals that could be identified as having been rated on both indices. Correlations between the six possible pairings of the four indices were tested with Spearman’s rho. Within each of the six possible pairings, the prevalence of identifiable errors was examined in a random selection of 10 journals and among the 10 most discordantly ranked journals on the two indices. The number of journals that could be matched within each pair of indices ranged from 1,857 to 6,508. Paired ratings for all journals showed strong to very strong correlations, with Spearman’s rho values ranging from 0.61 to 0.89, all p < 0.001. Identifiable errors were more common among scores for journals that had very discordant ranks on a pair of indices. These four journal citation indices were significantly correlated, providing evidence of convergent validity (i.e. they reflect the same underlying construct of average citability per article in a journal). Discordance in the ranking of a journal on two indices was in some cases due to an error in one index.     

Journal citation identity, journal citation image and internationalisation: Methods for journal evaluation

Summary Journal Citation Identity, Journal Citation Image, and Internationalisation are methods for journal evaluation used for an analysis of the Journal of Documentation(JDOC) which is compared to JASIS(T) and the Journal of Information Science(JIS). The set of analyses contributes to portrait a journal and gives a multifaceted picture. For instance, the Journal Citation Image by the New Journal Diffusion Factor tells that JDOC reaches farther out into the scientific community than the JASIS(T) and JIS. Comparing New Journal Diffusion Factor and Journal Impact Factor illustrates how new information has been added by the new indicator. Furthermore, JDOC is characterised by a higher rate of journal diversity in the references and has a lower number of scientific publications. JDOC authors and citers are affiliated Western European institutions at an increasing rate.     

Indices of journal citation relatedness and citation relationships among aquatic biology journals

Simple quantitative indices of pair-wise journal citation relatedness (based on the numbers of references given to and received from a journal title, which are provided by Science Citation Index database) are translated by an automatic clustering procedure into a meaningful map diagram reflecting topical relatedness of journals within a field of science. Such a map for 60 journals in marine and freshwater biology and related sciences published in 1987 reveals a tight cluster of marine biology journals quite distinct from the freshwater biology journal cluster and from the fisheries cluster. The journals within the marine biology cluster and those with strongest pair-wise links with them can be regarded as the core journals in marine biology. Indices of unilateral citation relatedness are used to obtain diagrams, which we term citograms. The citograms visualize patterns of citation relatedness of a journal (its citing and being cited). Journal self-citation can be meaningfully estimated using the bilateral index of relatedness. Self-citation is high in specialized or regional journal titles. It also appears to be quite substantial in journals of broader scope, which possibly reflect authors' subjective preferences.     

On generalising scientometric journal mapping beyond ISI's journal and citation databases

This article presents results of a study on the applicability of journal mapping of knowledge domains beyond the databases produced by the Institute for Scientific Information (ISI). The utility and validity of this generalisation is discussed with an emphasis on its added value in comparison to traditional ISI-based journal maps, i.e. those restricted to (predominantly English-language) ISI-covered journals, and journal-to-journal citation data retrieved from ISI'sJournal Citation Reports. The mapping methodology was applied to Manufacturing technology and management — a multidisciplinary domain situated on the interface of science and technology. TheInternational Journal of Production Economics was singled out as a special case for the validation study of the maps. Results of this study, involving several subject experts, indicate that a journal content-based map was not only far superior to the journal citation map, but also outperformed the map derived from the combination of both types of data. The selection of periodicals from other databases such as COMPENDEX produced a substantial number of additional titles of which only half were also covered by ISI.     

Examining national citation impact by comparing developments in a fixed and a dynamic journal set

Scientometrics - In order to examine potential effects of methodological choices influencing developments in relative citation scores for countries, a fixed journal set comprising of 3232 journals...     

Citation graph,weighted impact factors and performance indices

A scheme of evaluating an impact of a given scientific paper based on importance of papers quoting it is investigated. Introducing a weight of a given citation, dependent on the previous scientific achievements of the author of the citing paper, we define the weighting factor of a given scientist. Technically the weighting factors are defined by the components of the normalized leading eigenvector of the matrix describing the citation graph. The weighting factor of a given scientist, reflecting the scientific output of other researchers quoting his work, allows us to define weighted number of citation of a given paper, weighted impact factor of a journal and weighted Hirsch index of an individual scientist or of an entire scientific institution.     

Field normalized citation rates, field normalized journal impact and Norwegian weights for allocation of university research funds

We compared three different bibliometric evaluation approaches: two citation-based approaches and one based on manual classification of publishing channels into quality levels. Publication data for two universities was used, and we worked with two levels of analysis: article and department. For the article level, we investigated the predictive power of field normalized citation rates and field normalized journal impact with respect to journal level. The results for the article level show that evaluation of journals based on citation impact correlate rather well with manual classification of journals into quality levels. However, the prediction from field normalized citation rates to journal level was only marginally better than random guessing. At the department level, we studied three different indicators in the context of research fund allocation within universities and the extent to which the three indicators produce different distributions of research funds. It turned out that the three distributions of relative indicator values were very similar, which in turn yields that the corresponding distributions of hypothetical research funds would be very similar.     

A comparison of Australian university output using journal impact factors

We weighted the output of SCI items from Australian universities using journal impact factors. This provides us with an accessible quality indicator of science journal publishing, and allow us to scale for institutional size in terms of output and research staff. Use of this indicator for the 20 pre-1987 Australian universities demonstrates that although some universities rank highly on output, when scaled for institutional size they are overtaken by some of the smaller, more recently established universities.     

Reliability-based citation impact factor and the manipulation of impact factor

According to the definition of reliability-based citation impact factor (R-impact factor) proposed by KUO & RUPE and the cumulative citation age distribution model, a mathematical expression of the relationship between R-impact factor and impact factor is established in this paper. By simulation of the change processes of the R-impact factor and impact factor in the manipulation process of the impact factor, it is found that the effect of manipulation can be partly corrected by the R-impact factor in some cases. Based on the Journal Citation Report database, impact factors of 4 normal journals and 4 manipulated journals were collected. The journals’ R-impact factors and self-cited rates in the previous two years were calculated for each year during the period 2000 to 2007, and various characteristics influenced by the manipulation were analyzed. We find that the R-impact factor has greater fairness than the impact factor for journals with relatively short cited half-lives. Finally, some issues about using the R-impact factor as a measure for evaluating scientific journals are discussed.     

Normalisation of citation impact in economics

Scientometrics - This study is intended to facilitate fair research evaluations in economics. Field- and time-normalisation of citation impact is the standard method in bibliometrics. Since...     

Journal ranking by selective impact. New method based on SDI results and journal impact factors

Selecting an appropriate set of scientific journals which best meets the users' needs and the dynamics of science requires usage of weight parameters by which journals can be ranked. Previous methods are based on the simple counting of relevant articles, or hits in SDI runs. The new method proposed combines hit numbers in SDI runs and journals' impact factors to a weight parameter called Selective Impact. The experimental results obtained show that ranking by Selective Impact leads to a higher quality of the conclusions to be drawn from journal rank distributions.     

The generalized propensity score methodology for estimating unbiased journal impact factors

The journal impact factor (JIF) proposed by Garfield in the year 1955 is one of the most commonly used and prominent citation-based indicators of the performance and significance of a scientific journal. The JIF is simple, reasonable, clearly defined, and comparable over time and, what is more, can be easily calculated from data provided by Thomson Reuters, but at the expense of serious technical and methodological flaws. The paper discusses one of the core problems: The JIF is affected by bias factors (e.g., document type) that have nothing to do with the prestige or quality of a journal. For solving this problem, we suggest using the generalized propensity score methodology based on the Rubin Causal Model. Citation data for papers of all journals in the ISI subject category ??Microscopy?? (Journal Citation Report) are used to illustrate the proposal.     

Discovery of factors influencing citation impact based on a soft fuzzy rough set model

In this paper, the machine learning tools were used to identify key features influencing citation impact. Both the papers?? external and quality information were considered in constructing papers?? feature space. Based on the feature space, the soft fuzzy rough set was used to generate a series of associated feature subsets. Then, the KNN classifier was used to find the feature subset with the best classification performance. The results show that citation impact could be predicted by objectively assessed factors. Both the papers?? quality and external features, mainly represented as the reputation of the first author, are contributed to future citation impact.     

The multi-dimensionality of journal impact

In recent studies the issue of the relatedness between journal impact factors and other measures of journal impact have been raised and discussed from both merely empirical and theoretical perspectives. Models of the underlying citation processes suggest distributions with two or more free parameters. Proceeding from the relation between the journals’ mean citation rate and uncitedness and the assumption of an underlying Generalised Waring Distribution (GWD) model, it is found that the journal impact factor alone does not sufficiently describe a journal’s citation impact, while a two-parameter solution appropriately reflects its main characteristics. For the analysis of highly cited publications an additional model derived from the same GWD is suggested. This approach results in robust, comprehensible and interpretable solutions that can readily be applied in evaluative bibliometrics.     

Self-citation and citation of top journal publishers and their interpretation in the journal-discipline context

Scientometrics - The increasing influence of publishers on academic publishing has inevitably promoted studies of publisher citation and self-citation. However, in this type of citation analysis,...