Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis

During Eugene Garfield’s (EG’s) lengthy career as information scientist, he published about 1500 papers. In this study, we use the impressive oeuvre of EG to introduce a new type of bibliometric networks: keyword co-occurrences networks based on the context of citations, which are referenced in a certain paper set (here: the papers published by EG). The citation context is defined by the words which are located around a specific citation. We retrieved the citation context from Microsoft Academic. To interpret and compare the results of the new network type, we generated two further networks: co-occurrence networks which are based on title and abstract keywords from (1) EG’s papers and (2) the papers citing EG’s publications. The comparison of the three networks suggests that papers of EG and citation contexts of papers citing EG are semantically more closely related to each other than to titles and abstracts of papers citing EG. This result accords with the use of citations in research evaluation that is based on the premise that citations reflect the cognitive influence of the cited on the citing publication.     

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

     

From biomedical research to health improvement

Traditional means of analysis of research outputs have focussed on citations to papers in journals in other journal publications. But these only chronicle the early stages whereby research in biomedicine is converted into health improvement through better patient care and through preventive measures. New evaluation methods, still based on the concept of citation of research in other documents, are needed and are now being developed. These include the use of textbooks in medical education and the analysis of governmental regulations and health policies, which can influence both the availability of new drugs and the control of toxic substances in food and the environment. There is also an interest in the way that newspapers report biomedical research advances. Readers include politicians, healthcare professionals, the general public (who are increasingly becoming active consumers of healthcare products) and other researchers who may value the immediacy of the reporting. Newspaper articles tend to focus on fashionable topics and to offer premature hopes of cures to disease, but they can also provide a valuable service in showing the importance of animal experiments to biomedical progress. It would be useful to create an international database of newspaper citations through a consortium of partners in different countries who would agree a common protocol and exchange information on a regular basis. This revised version was published online in June 2006 with corrections to the Cover Date.     

Exploiting Structural Similarities to Classify Citations

Citations play an important role in the scientific community by assisting in measuring multifarious policies like the impact of journals, researchers, institutions, and countries. Authors cite papers for different reasons, such as extending previous work, comparing their study with the state-of-the-art, providing background of the field, etc. In recent years, researchers have tried to conceptualize all citations into two broad categories, important and incidental. Such a categorization is very important to enhance scientific output in multiple ways, for instance, (1) Helping a researcher in identifying meaningful citations from a list of 100 to 1000 citations (2) Enhancing the impact factor calculation mechanism by more strongly weighting important citations, and (3) Improving researcher, institutional, and university rankings by only considering important citations. All of these uses depend upon correctly identifying the important citations from the list of all citations in a paper. To date, researchers have utilized many features to classify citations into these broad categories: cue phrases, in-text citation counts, and metadata features, etc. However, contemporary approaches are based on identification of in-text citation counts, mapping sections onto the Introduction, Methods, Results, and Discussion (IMRAD) structure, identifying cue phrases, etc. Identifying such features accurately is a challenging task and is normally conducted manually, with the accuracy of citation classification demonstrated in terms of these manually extracted features. This research proposes to examine the content of the cited and citing pair to identify important citing papers for each cited paper. This content similarity approach was adopted from research paper recommendation approaches. Furthermore, a novel section-based content similarity approach is also proposed. The results show that solely using the abstract of the cited and citing papers can achieve similar accuracy as the state-of-the-art approaches. This makes the proposed approach a viable technique that does not depend on manual identification of complex features.     

Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles

Context. The use of citation frequency and impact factor as measures of research quality and journal prestige is being criticized. Citation frequency is augmented by self-citation and for most journals the majority of citations originate from a minority of papers. We hypothesized that citation frequency is also associated with the geographical origin of the research publication. Objective. We determined whether citations originate more frequently from institutes that are located in the same country as the authors of the cited publication than would be expected by chance. Design. We screened citations referring to 1200 cardiovascular publications in the 7 years following their publication. For the 1200 citation recipient publications we documented the country where the research originated (9 countries/regions) and the total number of received citations. For a selection of 8864 citation donor papers we registered the country/region where the citing paper originated. Results. Self-citation was common in cardiovascular journals (n = 1534, 17.8%). After exclusion of self-citation, however, the number of citations that originated from the same country as the author of the citation recipient was found to be on average 31.6% higher than would be expected by chance (p<0.01 for all countries/regions). In absolute numbers, nation oriented citation bias was most pronounced in the USA, the country with the largest research output (p<0.001). Conclusion. Citation frequency was significantly augmented by nation oriented citation bias. This nation oriented citation behaviour seems to mainly influence the cumulative citation number for papers originating from the countries with a larger research output.     

Using categorisations of citations when assessing the outcomes from health research

Summary This paper describes an attempt to explore how far a categorisation of citations could be used as part of an assessment of the outcomes from health research. A large-scale project to assess the outcomes from basic, or early clinical, research is being planned, but before proceeding with such a project it was thought important to test and refine the developing methods in a preliminary study. Here we describe the development, and initial application, of one element of the planned methods: an approach to categorising citations with the aim of tracing the impact made by a body of research through several generations of papers. The results from this study contribute to methodological development for the large-scale project by indicating that: only for a small minority of citing papers is the cited paper of considerable importance; the number of times a paper is cited can not be used to indicate the importance of that paper to the articles that cite it; and self-citations could play an important role in facilitating the eventual outcomes achieved from a body of research.     

Citations and certainty: a new interpretation of citation counts

Scientometrics - We report that the rate of hedging in citing sentences for biomedical papers is inversely related to the citations received by the papers as measured by the number of citances in...     

CAD: an algorithm for citation-anchors detection in research papers

Citations are very important parameters and are used to take many important decisions like ranking of researchers, institutions, countries, and to measure the relationship between research papers. All of these require accurate counting of citations and their occurrence (in-text citation counts) within the citing papers. Citation anchors refer to the citation made within the full text of the citing paper for example: ‘[1]’, ‘(Afzal et al, 2015)’, ‘[Afzal, 2015]’ etc. Identification of citation-anchors from the plain-text is a very challenging task due to the various styles and formats of citations. Recently, Shahid et al. highlighted some of the problems such as commonality in content, wrong allotment, mathematical ambiguities, and string variations etc in automatically identifying the in-text citation frequencies. The paper proposes an algorithm, CAD, for identification of citation-anchors and its in-text citation frequency based on different rules. For a comprehensive analysis, the dataset of research papers is prepared: on both Journal of Universal Computer Science (J.UCS) and (2) CiteSeer digital libraries. In experimental study, we conducted two experiments. In the first experiment, the proposed approach is compared with state-of-the-art technique over both datasets. The J.UCS dataset consists of 1200 research papers with 16,000 citation strings or references while the CiteSeer dataset consists of 52 research papers with 1850 references. The total dataset size becomes 1252 citing documents and 17,850 references. The experiments showed that CAD algorithm improved F-score by 44% and 37% respectively on both J.UCS and CiteSeer dataset over the contemporary technique (Shahid et al. in Int J Arab Inf Technol 12:481–488, 2014). The average score is 41% on both datasets. In the second experiment, the proposed approach is further analyzed against the existing state-of-the-art tools: CERMINE and GROBID. According to our results, the proposed approach is best performing with F1 of 0.99, followed by GROBID (F1 0.89) and CERMINE (F1 0.82).     

Citation context analysis of a co-citation cluster: Recombinant-DNA

The techniques of co-citation clustering and citation context analysis are combined to concretely define the shared knowledge within a research specialty. The cluster for a large and fast moving biomedical specialty, recombinant-DNA, is presented in terms of the highly cited documents comprising it and their co-citation links. By examining citation contexts in the papers citing the highly cited documents, it is possible to label each of the documents in the cluster with its specific cognitive meaning for the citing authors. Co-citation contexts are used to reveal the relationships among the concepts symbolized by the highly cited documents, providing a cognitive equivalent of the co-citation links. This may open a new way to the investigation of the logic of conceptual change at the specialty level.     

The citation advantage of foreign language references for Chinese social science papers

Contemporary scientific exchanges are international, yet language continues to be a persistent barrier to scientific communication, particularly for non-native English-speaking scholars. Since the ability to absorb knowledge has a strong impact on how researchers create new scientific knowledge, comprehensive access to and understanding of both domestic and international scientific publications is essential for scientific performance. This study explores the effect of absorbed knowledge on research impact by analyzing the relationship between the language diversity of cited references and the number of citations received by the citing paper. Chinese social sciences are taken as the research object, and the data, 950,302 papers published between 1998 and 2013 with 8,151,327 cited references, were collected from the Chinese Social Sciences Citation Index. Results show that there is a stark increase in the consumption of foreign language material within the Chinese social science community, and English material accounts for the vast majority of this consumption. Papers with foreign language references receive significantly more citations than those without, and the citation advantage of these internationalized work holds when we control for characteristics of the citing papers, such as the discipline, prestige of journal, prestige of institution, and scientific collaboration. However, the citation advantage has decreased from 1998 to 2008, largely as an artifact of the increased number of papers citing foreign language material. After 2008, however, the decline of the citation advantage subsided and became relatively stable, which suggests that incorporating foreign language literature continues to increase scientific impact, even as the scientific community itself becomes increasingly international. However, internationalization is not without concerns: the work closes with a discussion of the potential problems associated with the lack of linguistic diversity in scientific communication.

     

Is citation analysis a legitimate evaluation tool?

A comprehensive discussion on the use of citation analysis to rate scientific performance and the controversy surrounding it. The general adverse criticism that citation counts include an excessive number of negative citations (citations to incorrect results worthy of attack), self-citations (citations to the works of the citing authors), and citations to methodological papers is analyzed. Included are a discussion of measurement problems such as counting citations for multiauthored papers, distinguishing between more than one person with the same last name (homographs), and what it is that citation analysis actually measures. It is concluded that as the scientific enterprise becomes larger and more complex, and its role in society more critical, it will become more difficult, expensive and necessary to evaluate and identify the largest contributors. When properly used, citation analysis can introduce a useful measure of objectivity into the evaluation process at relatively low financial cost.Modification of a chapter in E. Garfield:Citation Indexing: Its Theory and Application in Science, Technology and the Humanities, New York, Wiley, 1979.     

Influence of proceedings papers on citation impact in seven sub-fields of sustainable energy research 2005–2011

This paper analyses the following seven sub-fields of Sustainable Energy Research with respect to the influence of proceedings papers on citation patterns across citing and cited document types, overall sub-field and document type impacts and citedness: the Wind Power, Renewable Energy, Solar and Wave Energy, Geo-thermal, Bio-fuel and Bio-mass energy sub-fields. The analyses cover peer reviewed research and review articles as well as two kinds of proceeding papers from conferences published 2005–2009 in (a) book series or volumes and (b) special journal issues excluding meeting abstracts cited 2005–2011 through Web of Science. Central findings are: The distribution across document types of cited versus citing documents is highly asymmetric. Predominantly proceedings papers from both proceeding volumes as well as published in journals cite research articles (60–76 %). Largely, journal-based proceedings papers are cited rather than papers published in book series or volumes and have field impacts corresponding to research articles. With decreasing proceedings paper dominance in research fields the ratio of proceeding paper volumes over journal-based proceedings papers decreases significantly and the percentage of proceedings papers in journals citing journal-based proceedings papers over all publications citing journal-based proceedings papers decreases significantly (from 26.3 % in Wind Power to 4 % in Bio Fuel). Further, the segment of all kinds of proceedings papers (the combined proceedings paper types) citing all proceedings papers over all publications citing all kinds of proceedings papers decreases significantly (from 36.1 % in Wind Power to 11.3 % in Bio Fuel). Simultaneously the field citedness increases across the seven research fields. The distribution of citations from review articles shows that novel knowledge essentially derives directly from research articles (53–72 %)—to a much less extent from proceedings publications published in journals (9–13 %).     

A longitudinal analysis of citation distribution breadth for Chinese scholars

Over the past 30 years, the research behavior of Chinese scholars has continually evolved. This paper studied the citing behavior of Chinese scholars by employing three indicators of citation concentration from the perspective of citation breadth analysis. All the citations from 2,338,033 papers from the Chinese Citation Database (1979–2008) covering four disciplines—Chemistry; Clinical Medicine; Library, Information and Archival Science; and Chinese Literature and World Literature—were analyzed. Empirical results show a general weakening tendency towards citation concentration: (1) decreasing percentage of uncited published papers within a given year; (2) a higher percentage of papers required to account for the same proportion of citation than before; and (3) the steady decline in the Herfindahl-Hirschman index (HHI) of citation distribution. All three measures indicate a decline in citing concentration or an increase in citation breadth. This phenomenon may be the result of increased access to materials, perhaps because of the ease with which scholarly materials can be accessed through the Internet.     

A quasi-quantitative citation model

On the basis of investigating author's opinion on citing motivations of chemistry papers aquasi-quantitative model for citing is suggested. The model selects professional and nonprofessional motivations of citing and introduces thecitation threshold concept which tries to characterize the effect of citing motivations quantitatively. Possible reasons for missing citations are also treated. Mean ages of real and of self-citations were calculated by subtracting the average of the publication years of cited papers from the publication year of the citing publication. The difference between the mean ages may characterize thesynchronity of the author's research in comparison with those working on similar topics. The paper introduces thecitation strategy indicator which relates impact factors of cited periodicals with the mean impact factor of periodicals in the corresponding research subfield.     

Publication and citation patterns of the Mexican contribution to a “Big Science” discipline: Elementary particle physics

The publication and citation patterns of the Mexican community in elementary particle physics (MEPP) were determined by bibliometric analysis of the scientific production and citations registered in the SPIRES-HEP system from 1971 to 2000. All papers, both citing and cited, were classified as theoretical, phenomenological or experimental according to the type of study carried out and citing papers as local (Mexican) or foreign. The growth dynamics of the citation patterns over the thirty-year period was also studied. Results show that the Mexican scientific community in EPP follow the pre-publication and pre-citation communication patterns typical of a Big Science field. This revised version was published online in June 2006 with corrections to the Cover Date.     

Exploring the analytical potential of comparing citing and cited source items

Summary Comparing properties of citing and cited source items opens a wide variety of analytical possibilities. In a study of citations among papers in the journal Scientometrics a number of analytical themes are identified. The analysis shows: the way in which a citation graph can be decomposed into different subparts; country specific citation patterns; the effects of self-citations and domestic citations; the mapping of cited author relationships using direct citation and co-citation links; and time slicing effects on impact ranking of countries and papers.     

Citation time window choice for research impact evaluation

This paper aims to inform choice of citation time window for research evaluation, by answering three questions: (1) How accurate is it to use citation counts in short time windows to approximate total citations? (2) How does citation ageing vary by research fields, document types, publication months, and total citations? (3) Can field normalization improve the accuracy of using short citation time windows? We investigate the 31-year life time non-self-citation processes of all Thomson Reuters Web of Science journal papers published in 1980. The correlation between non-self-citation counts in each time window and total non-self-citations in all 31 years is calculated, and it is lower for more highly cited papers than less highly cited ones. There are significant differences in citation ageing between different research fields, document types, total citation counts, and publication months. However, the within group differences are more striking; many papers in the slowest ageing field may still age faster than many papers in the fastest ageing field. Furthermore, field normalization cannot improve the accuracy of using short citation time windows. Implications and recommendations for choosing adequate citation time windows are discussed.     

Significant difference of dependence upon scientific knowledge among different technologies

Summary The authors have constructed an original database of the full text of the Japanese Patent Gazette published since 1994. The database includes not only the front page but also the body text of more than 880,000 granted Japanese patents. By reading the full texts of all 1,500 patent samples, we found that some inventors cite many academic papers in addition to earlier patents in the body texts of their Japanese patents. Using manually extracted academic paper citations and patent citations as “right” answers, we fine-tuned a search algorithm that automatically retrieves cited scientific papers and patents from the entire texts of all the Japanese patents in the database. An academic paper citation in a patent text indicates that the inventor used scientific knowledge in the cited paper when he/she invented the idea codified in the citing patent. The degree of science linkage, as measured by the number of research papers cited in patent documents, is particularly strong in biotechnology. Among other types of technology, those related to photographic-sensitized material, cryptography, optical computing, and speech recognition also show strong science linkage. This suggests that the degree of dependence on scientific knowledge differs from technology to technology and therefore, different ways of university-industry collaboration are necessary for different technology fields.