Some methodological problems in ranking scientists by citation analysis

A sample of 80 Hungarian scientists, authors or co-authors of a total number of 6273 papers—published between 1930–1976—has been analysed. Citation data to eachpaper were collected form the 1964–76 SCI's by manual search. Citation counts were distinguished with respect to the following categories: (I) the set of cited authors has element(s) common with the set of citing authors (self citation), (II) condition I is not satisfied, but the cited author under study and at least one of the citing authors were co-authors prior to the publication of the cited paper, (III) none of the former criteria is satisfied. The yearly average citation frequency of a paper was not corrected for obsolescence, since there is no evidence that the decay of citation frequency with time is independent of the absolute citedness of the paper. Individual performance has been measured (a) by the sum of the vearly average typeIII fractional citation frequencies over all of the author's papers, (b) by the sum of the yearly average citation frequency normalized to one single-authored paper per year over the period of the author's activity, (c) by the same as ina, but summed up only over the most highly cited papers scattering upwards from the individual's own average, (d) by the fractional authorship, and (e) by the number of items in the author's publication list. The first three parameters seem to be applicable in measuring the utility of the individual's scientific contribution with slightly different emphasis on different aspects. These parameters are uncorrelated with those measuring the output of individuals.     

Literature retrieval based on citation context

While the citation context of a reference may provide detailed and direct information about the nature of a citation, few studies have specifically addressed the role of this information in retrieving relevant documents from the literature primarily due to the lack of full text databases. In this paper, we design a retrieval system based on full texts in the PubMed Central database. We constructed two modules in the retrieval system. One is a reference retrieval module based on citation contexts. Another is a citation context retrieval module for searching the citation contexts of a specific paper. The results of comparisons show that the reference retrieval module performed better than Google Scholar and PubMed database in terms of finding proper references based on topic words extracted from citation context. It also performed very well on searching highly cited papers and classic papers. The citation context retrieval module visualizes the topics of citation contexts as tag clouds and classifies citation contexts based on cue words in citation contexts.     

Are classic references cited first? An analysis of citation order within article sections

Scientometrics - Early citations within an article section may have an agenda-setting role but contribute little to the new research. To investigate whether this practice may be common, this...     

Understanding the impact change of a highly cited article: a content-based citation analysis

Researchers tend to cite highly cited articles, but how these highly cited articles influence the citing articles has been underexplored. This paper investigates how one highly cited essay, Hirsch’s “h-index” article (H-article) published in 2005, has been cited by other articles. Content-based citation analysis is applied to trace the dynamics of the article’s impact changes from 2006 to 2014. The findings confirm that citation context captures the changing impact of the H-article over time in several ways. In the first two years, average citation mention of H-article increased, yet continued to decline with fluctuation until 2014. In contrast with citation mention, average citation count stayed the same. The distribution of citation location over time also indicates three phases of the H-article “Discussion,” “Reputation,” and “Adoption” we propose in this study. Based on their locations in the citing articles and their roles in different periods, topics of citation context shifted gradually when an increasing number of other articles were co-mentioned with the H-article in the same sentences. These outcomes show that the impact of the H-article manifests in various ways within the content of these citing articles that continued to shift in nine years, data that is not captured by traditional means of citation analysis that do not weigh citation impacts over time.     

The influence of Elfreda Chatman’s theories: a citation context analysis

The range of theoretical frameworks currently being used by researchers into information behaviour is abundant and diverse. We need to examine thoroughly the contribution of theories and models to further research, as this would help to improve future investigations in the field. This paper adopts this approach, by thoroughly examining the influence that Elfreda Chatman’s three middle-range theories have had on subsequent research. A citation context analysis was carried out on the basis of those received by Information poverty theory, life in the round theory and normative behaviour theory. Analysis covered the year of publication, the type of work and the subject-matter of the citing documents. The cites in context or theoretical incidents were analysed for frequency of citation in citing documents, the content of Chatman’s work being cited, the context co-citation analysis, the citation style and the citation location. The analysis of citation in context has allowed us to draw a distinction between the author and her work, while verifying that not all cites are the same. These differences reflect the unequal relevance of these theories to subsequent research.

     

A generative model of article citation networks of a subject from a large-scale citation database

Scientometrics - In this paper, we analyze the structure of the article citation network of a particular subject obtained from the Web of Science (WoS) database. In specific, we modify a model...     

Online citation analysis

The paper investigates the online citation analysis possibilities and limitations. The following online processing tools: RANK, MAP, and TARGET, provided by Dialog, are incorporated in order to perform analyses of citations to and from isolated sets of documents as well as to carry out diachrone journal analyses. These, analyses imply further to determine journal impact factors of ISI journals. Measures of the scope of internationalisation of journals are proposed and demonstrated. By the combined application of the RANK and TARGET commands we demonstrate a hitherto overlooked possibility of working with bibliographic coupling online and mapping of scientific fields.     

Patterns of annual citation of highly cited articles and the prediction of their citation ranking: A comparison across subjects

High citation is associated with research quality and consequently findings on highly cited articles are useful to increase understanding of the factors that produce high quality research. This study explores highly cited articles in six subjects, focusing on late citation and peak citation years. Longitudinal citation patterns were found to be highly varied and, on average, different from the remaining articles in each subject. For four of the six subjects, there is a correlation of over 0.42 between the percentage of early citations and total citation ranking but more highly ranked articles had a lower percentage of early citations. Surprisingly, for highly cited articles in all six subjects the prediction of citation ranking of from the sum of citations during their first six years was less accurate than prediction using the sum of the citations for only the fifth and sixth year.     

A review of citation recommendation: from textual content to enriched context

Scientometrics - Citation recommendation systems play an important role to alleviate the dilemma that scholar users spend a lot of time and experiences for literature survey. With the burgeoning...     

Interpreting maps of science using citation context sentiments: a preliminary investigation

It is proposed that citation contexts, the text surrounding references in scientific papers, be analyzed in terms of an expanded notion of sentiment, defined to include attitudes and dispositions toward the cited work. Maps of science at both the specialty and global levels are used as the basis of this analysis. Citation context samples are taken at these levels and contrasted for the appearance of cue word sets, analyzed with the aid of methods from corpus linguistics. Sentiments are shown to vary within a specialty and can be understood in terms of cognitive and social factors. Within-specialty and between-specialty co-citations are contrasted and in some cases suggest a correlation of sentiment with structural location. For example, the sentiment of “uncertainty” is important in interdisciplinary co-citation links, while “utility” is more prevalent within the specialty. Suggestions are made for linking sentiments to technical terms, and for developing sentiment “baselines” for all of science.     

Patent citation analysis: A policy analysis tool

Patent citation analysis is a recent development which uses bibliometric techniques to analyse the wealth of patent citation information. This paper describes the various facets of patent citations and patent citation studies, and their important applications. Construction of technology indicators being an important use of patent citations, various patent citation based technological indicators and their applications are also described.     

A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking

Two paradigmatic approaches to the normalisation of citation-impact measures are discussed. The results of the mathematical manipulation of standard indicators such as citation means, notably journal Impact Factors, (called a posteriori normalisation) are compared with citation measures obtained from fractional citation counting (called a priori normalisation). The distributions of two subfields of the life sciences and mathematics are chosen for the analysis. It is shown that both methods provide indicators that are useful tools for the comparative assessment of journal citation impact.     

Extending citation analysis to non-source items

Summary This paper reports the first results of the extension of citation analysis to 'non-source' items, which is one strand of an extensive study of quantitative performance indicators used in the assessment of research. It would be presumptuous to draw firm conclusions from this first foray into the realm of non-source citations, however our analysis is based on an extensive experimental database of over 30,000 publications, so the results can be viewed as strong pointers to possible generalised outcomes. We show that it is possible to mine ISI databases for references to a comprehensive oeuvre of items from whole institutions. Many types of publications are visible in the ISI data - books, book chapters, journals not indexed by ISI, and some conference publications. When applied to the assessment of university departments, they can have a significant effect on rankings, though this does not follow in all cases. The investment of time, effort, and money in a significantly extended analysis will not be equally beneficial in all fields. However, a considerable amount of testing is required to confirm our initial results.     

Article-by-article citation analysis of medical journals

An article by article analysis produced by ISI has been investigated to see whether this form of feedback might be useful to the editors. The data highlight the different roles of two medical journals, which are often regarded as similar. They also allow a parallel examination of the citation pattern of other items besides the standard scientific research articles.     

The impact of qualitative methods on article citation: an international business research perspective

Scientometrics - Qualitative methods have traditionally been underused in international business, and their research potential in this area has not been widely investigated. The main objective of...     

A compendium of issues for citation analysis

This paper examines a number of the criticisms that citation analysis has been subjected to over the years. It is argued that many of these criticisms have been based on only limited examinations of data in particular contexts and it remains unclear how broadly applicable these problems are to research conducted at different levels of analysis, in specific field, and among various national data sets. Relevant evidence is provided from analysis of Australian and international data. Citation analysis is likely to be most reliable when data is aggregated and at the highly-cited end of the distribution. It is possible to make valid inferences about individual cases, although considerable caution should be used. Bibliometric measures should be viewed as a useful supplement to other research evaluation measures rather than as a replacement.     

Sample size bounding and context ranking as approaches to the HRA data problem

This paper presents a technique denoted as sub sample size bounding (SSSB) useable for the statistical derivation of context-specific probabilities from data available in existing reports on operating experience. Applications for human reliability analysis (HRA) are emphasized in the presentation of the technique. Exemplified by a sample of 180 abnormal event sequences, it is outlined how SSSB can provide viable input for the quantification of errors of commission (EOCs).