Publication types,citation rates and evaluation

In order to resolve questions frequently raised in the context of research evaluation about the citation rates of journal publications in relation to other types of publications, the total research output of substantial institutions or systems has to be brought under bibliographic control. That precondition has rarely been met: there are few published studies of the total range of publications of major research institutions, including books, book chapters, technical reports and published conference proceedings. The Research Evaluation and Policy Project (REPP) at the Australian National University (ANU) has established a database covering all the publications from the Institute of Advanced Studies (IAS), a fulltime research institution at the ANU, and has examined in detail citations in the journal literature accruing to all types of publications. The database contains a significant number of publications, nearly 30 000 items, and covers the sciences and the social sciences and humanities. This data enables us to examine whether the citation record of research publications appearing in journals indexed by the Institute for Scientific Information (ISI) is a useable surrogate for the citation record within ISI journals of all model of publication. We contend that, if certain preconditions are met, the choice of citation rate is not critical.     

Are academics who publish more also more cited? Individual determinants of publication and citation records

Thanks to a unique individual dataset of French academics in economics, we explain individual publication and citation records by gender and age, co-authorship patterns (average number of authors per article and size of the co-author network) and specialisation choices (percentage of output in each JEL code). The analysis is performed on both EconLit publication scores (adjusted for journal quality) and Google Scholar citation indexes, which allows us to present a broad picture of knowledge diffusion in economics. Citations are largely driven by publication records, which means that these two measures are partly substitutes, but citations are also substantially increased by larger research team size and co-author networks.     

International and domestic co-publishing and their citation impact in different disciplines

This paper studies disciplinary differences in citation impacts of different types of co-publishing. The citation impacts of international, domestic inter-organizational and domestic intra-organizational co-publications, and single-authored publications, are compared. In particular, we examine the extent to which the number of authors explains the potential differences in citation impacts when compared to the influence of different types of international and domestic collaborations. The analysis is based on Finland’s publications in Thomson Reuters Web of Science database in 1990–2008. Finland is a small country, thus, it has fewer opportunities to find collaborators inside own country when compared to larger countries. Finland’s science policy has underlined internationalization and research collaboration as key means to increase the quality and impact of Finnish research. This study indicates that both international and domestic co-publishing have steadily increased during the past two decades in all disciplinary groups. International co-publications gain on average more citations than domestic co-publications. In natural sciences and engineering, co-authorship explains only a small proportion of variability in publications’ citation rates. When the effect of the number of authors is taken into account there are no big differences in citation impacts between international and domestic co-publications. However, international co-publications by ten authors or more gather significantly more citations than other publications. In humanities, the difference in citation impacts between co-authored publications in relation to single-authored publications is significant. However, international co-publications are not on average more highly cited in relation to domestic co-publications in humanities.     

Use of bibliometric information to assist research policy making. A comparison of publication and citation profiles of Full and Associate Professors at a School of Chemistry in Uruguay

Summary Publication and citation profiles of Full and Associate Professors at the School of Chemistry of the Universidad de la República in Uruguay were investigated. The groups do not exhibit markedly different age averages. However, the average time since they started publishing, as well as other characteristics of their publication records, like productivity or citations, set them apart. From the point of view of both the number of papers per author and per year of activity, on one side, and of the number of citations per year of activity, on the other, the group of Full Professors has statistically significant larger averages than the Associate Professors. The impact of self-citations, multi-authorship and internationalization of the publications were analyzed within the two groups and shown to have no excessive or predictable influence on those parameters, except in the case of few (≤ 2) or many (>8) authors. It is suggested in this paper that these two indicators, number of papers per author per production year and number of citations per production year, combined in a plot allowing a bidimensional ranking of the individuals in the groups, may be used profitably as one of the components in the development of a policy toward promotion of Associate Professors. The analysis showed also that the quotient of citations received to number of papers published, even when derived from actual citation data of the scientists without involving the impact factors of the journals in which they publish, are not good parameters to use for that purpose, essentially because there is a reduction in the information content of the indicator with respect to those described before.     

Relationship among research collaboration, number of documents and number of citations: a case study in Spanish computer science production in 2000–2009

This paper analyzes the relationship among research collaboration, number of documents and number of citations of computer science research activity. It analyzes the number of documents and citations and how they vary by number of authors. They are also analyzed (according to author set cardinality) under different circumstances, that is, when documents are written in different types of collaboration, when documents are published in different document types, when documents are published in different computer science subdisciplines, and, finally, when documents are published by journals with different impact factor quartiles. To investigate the above relationships, this paper analyzes the publications listed in the Web of Science and produced by active Spanish university professors between 2000 and 2009, working in the computer science field. Analyzing all documents, we show that the highest percentage of documents are published by three authors, whereas single-authored documents account for the lowest percentage. By number of citations, there is no positive association between the author cardinality and citation impact. Statistical tests show that documents written by two authors receive more citations per document and year than documents published by more authors. In contrast, results do not show statistically significant differences between documents published by two authors and one author. The research findings suggest that international collaboration results on average in publications with higher citation rates than national and institutional collaborations. We also find differences regarding citation rates between journals and conferences, across different computer science subdisciplines and journal quartiles as expected. Finally, our impression is that the collaborative level (number of authors per document) will increase in the coming years, and documents published by three or four authors will be the trend in computer science literature.     

A bibliometric analysis of physics publications in Korea, 1994-1998

This study examined research performance of Korean physicists, comparing Korean-authoredpapers versus internationally co-authored papers, indexed in SCI, 1994-1998, and using thenumber of citations received by internationally co-authored papers covered by the SCI CD-ROM.For the study, 4,665 papers published from the researchers affiliated with the physics departmentsor physics-associated laboratories at Korean universities and indexed by SCI were analyzed.Korean authored papers tended to be published in Korean, Japanese, and UK journals, whileinternationally co-authored papers were more likely to appear in German, Dutch, and Swissjournals. Among the 18 authorship countries (on the basis of first author), 93 internationally co-authored papers by U.S. researchers had the highest citation rate, an average 15.9 citations perpaper. Of the eight countries that published over 5 papers, there was no correlation between theaverage number of citations per paper and the total number of citations. However, an ANOVAindicated a significant difference between the average number of citations per paper according tocountry (F = 5.84, p < 0.0005). In other words, papers by the U.S. and French researchers tendedto be cited more frequently than papers by the Italian, Japanese, Korean, Russian, and Germanresearchers.     

Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles?

An analysis of 2, 765 articles published in four math journals from 1997 to 2005 indicate that articles deposited in the arXiv received 35% more citations on average than non-deposited articles (an advantage of about 1.1 citations per article), and that this difference was most pronounced for highly-cited articles. Open Access, Early View, and Quality Differential were examined as three non-exclusive postulates for explaining the citation advantage. There was little support for a universal Open Access explanation, and no empirical support for Early View. There was some inferential support for a Quality Differential brought about by more highly-citable articles being deposited in the arXiv. In spite of their citation advantage, arXiv-deposited articles received 23% fewer downloads from the publisher’s website (about 10 fewer downloads per article) in all but the most recent two years after publication. The data suggest that arXiv and the publisher’s website may be fulfilling distinct functional needs of the reader.     

A comparison of the scientific performance of the U.S. and the European union at the turn of the 21st century

In this paper, scientific performance is identified with the impact that journal articles have through the citations they receive. In 15 disciplines, as well as in all sciences as a whole, the EU share of total publications is greater than that of the U.S. However, as soon as the citations received by these publications are taken into account the picture is completely reversed. Firstly, the EU share of total citations is still greater than the U.S. in only seven fields. Secondly, the mean citation rate in the U.S. is greater than in the EU in every one of the 22 fields studied. Thirdly, since standard indicators—such as normalized mean citation ratios—are silent about what takes place in different parts of the citation distribution, this paper compares the publication shares of the U.S. and the EU at every percentile of the world citation distribution in each field. It is found that in seven fields the initial gap between the U.S. and the EU widens as we advance towards the more cited articles, while in the remaining 15 fields—except for Agricultural Sciences—the U.S. always surpasses the EU when it counts, namely, at the upper tail of citation distributions. Finally, for all sciences as a whole the U.S. publication share becomes greater than that of the EU for the top 50% of the most highly cited articles. The data used refers to 3.6 million articles published in 1998–2002, and the more than 47 million citations they received in 1998–2007.     

Do types of collaboration change citation? Collaboration and citation patterns of South African science publications

Bibliographic records are extensively used in the study of citations. Based on ISI data, this paper examines citation patterns of the publications of South African scientists in recent years. In particular, the focus of this paper is on citations as to the collaborative dimensions of South African scientists in their publications. The study reveals that the number of citations received by a publication varies not only according to the collaboration but also to the types of collaboration of the authors who are involved in its production. Furthermore, it emerges that the impact of citations on publications differs from discipline to discipline, and affiliating sector to sector, regardless of collaboration.     

Scientometrics of laser research in India during 1970-1994

An analysis of 952 publications published by Indian scientists and abstracted by Journal of Current Laser Abstracts during 1970-1994 indicates that laser research in India picked up during 1978-1994 and reached its peak in 1980. The Indian output in the field of laser research forms an integral part of the mainstream science as reflected by the pattern of publications and their citations in the international literature. Laser research performed in India improved considerably during 1985-1994 as compared to 1970-1984 as seen by different impact indicators such as citation per paper, proportion of high quality papers, and publication effective index. The publication output is concentrated among few institutions and there is a similarity in the activity and attractively profile of the highly productive institutions. India"s citation rate per paper for highly productive authors is at par with the world citation rate per paper. The study indicates that the proportion of mega authored papers increased during 1990-1994 and the international collaboration is mainly with the USA.     

Science in Scandinavia: A Bibliometric Approach

The development of publication activity and citation impact in Scandinavian countries is studied for the 1980–1997 period. Besides the analysis of trends in publication and citation patterns and of national publication profiles, an attempt is made to find statistical evidences of the relation between international co-authorship and both research profile and citation impact in the Nordic countries. A coherent Scandinavian cluster has been found, and the Nordic countries have strong co-authorship links with highly developed countries in West Europe and North America. It was found that international co-authorship, in general, results in publications with higher citation rates than purely domestic papers. International collaboration has, however, not the same influence on publication profiles and citation impact of each analysed countries.     

Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles

Context. The use of citation frequency and impact factor as measures of research quality and journal prestige is being criticized. Citation frequency is augmented by self-citation and for most journals the majority of citations originate from a minority of papers. We hypothesized that citation frequency is also associated with the geographical origin of the research publication. Objective. We determined whether citations originate more frequently from institutes that are located in the same country as the authors of the cited publication than would be expected by chance. Design. We screened citations referring to 1200 cardiovascular publications in the 7 years following their publication. For the 1200 citation recipient publications we documented the country where the research originated (9 countries/regions) and the total number of received citations. For a selection of 8864 citation donor papers we registered the country/region where the citing paper originated. Results. Self-citation was common in cardiovascular journals (n = 1534, 17.8%). After exclusion of self-citation, however, the number of citations that originated from the same country as the author of the citation recipient was found to be on average 31.6% higher than would be expected by chance (p<0.01 for all countries/regions). In absolute numbers, nation oriented citation bias was most pronounced in the USA, the country with the largest research output (p<0.001). Conclusion. Citation frequency was significantly augmented by nation oriented citation bias. This nation oriented citation behaviour seems to mainly influence the cumulative citation number for papers originating from the countries with a larger research output.     

Research performance of Turkish astronomers in the period of 1980–2010

We investigated the development of astronomy and astrophysics research productivity in Turkey in terms of publication output and their impacts as reflected in the science citation index for the period 1980–2010. Our study involves 838 refereed publications, including 801 articles, 16 letters, 15 reviews, and six research notes. The number of papers were prominently increased after 2000 and the average number of papers per researcher is calculated as 0.89. Total number of received citations for 838 papers is 6938, while number of citations per papers is approximately 8.3 in 30 years. Publication performance of Turkish astronomers and astrophysicists was compared with those of seven countries that have similar gross domestic expenditures on research and development, and members of organization for economic co-operation and development. Our study reveals that the output of astronomy and astrophysics research in Turkey has gradually increased over the years.     

Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis

During Eugene Garfield’s (EG’s) lengthy career as information scientist, he published about 1500 papers. In this study, we use the impressive oeuvre of EG to introduce a new type of bibliometric networks: keyword co-occurrences networks based on the context of citations, which are referenced in a certain paper set (here: the papers published by EG). The citation context is defined by the words which are located around a specific citation. We retrieved the citation context from Microsoft Academic. To interpret and compare the results of the new network type, we generated two further networks: co-occurrence networks which are based on title and abstract keywords from (1) EG’s papers and (2) the papers citing EG’s publications. The comparison of the three networks suggests that papers of EG and citation contexts of papers citing EG are semantically more closely related to each other than to titles and abstracts of papers citing EG. This result accords with the use of citations in research evaluation that is based on the premise that citations reflect the cognitive influence of the cited on the citing publication.     

A study on the citation situation within the citing paper: citation distribution of references according to mention frequency

Citation impact indicators play a significant role in evaluating the scientific research activity. Most of citation impact indicators are based on the citation count that the publication is cited as a reference in the other publications, but the difference between each citation situation was not considered. Normally, the number of citations that a publication is cited in the other publications may represent the formal quality of the publication. Similarly, the number of times that a publication is really mentioned within the citing publication, it may also represent the formal quality of the citation. We have examined about how many times each reference was really mentioned within the citing publications and studied about the citation situation within the citing publications. We verified that the citation distribution of references according to the mention frequency follows the Generalized Pareto distribution. The results showed that about 20% of total references were mentioned three and more times, and the number of citation mentions for the about 50% of total references were from about 20% of the total references in the citing publications.     

Changing trends in publishing behaviour among university faculty, 1980-2000

This article analyses changes in publication patterns over a twenty-year period at Norwegian universities. Based on three surveys among academic staff; in 1982, 1992 and 2001, covering all kinds of publications, the following general conclusions are drawn: (a) co-authorship has become more common, (b) the extent of publishing directed towards an international audience has increased, (c) the scientific article in an international journal has enhanced its position as the dominating type of publication, and (d) the number of publications per academic staff member has increased. The largest changes have taken place within the social sciences, which to an increasing extent approaches the publication pattern in the natural sciences. On the other hand, the large productivity differences between individual researchers have remained remarkably stable over the two decades in all fields of learning. This revised version was published online in August 2006 with corrections to the Cover Date.     

Research and citation impact of publications by the Chemistry Division at Bhabha Atomic Research Centre

The paper analyses the citations to 1733 publications published during 1970–1999 by the Chemistry Division at Bhabha Atomic Research Centre, using Science Citation Index 1982–2003 as the source data. The extent of citations received, in terms of the number of citations per paper, yearwise break up of citations, domainwise citations, self-citations and citations by others, diachronous self-citation rate, citing authors, citing institutions, highly cited papers, the categories of citing documents, citing journals and distribution of citations among them etc. are determined. During 1982–2003 chemistry Division publications have received a total of 11041 citations. The average number of citations per year was 501.86. The average number of citations per publication was 6.37. The highest number of citations received were 877 in 2001. The citation rate was peaked during 1990–2003 as maximum 9145 (82.82%) citations were received during the period. Total self-citations were 3716 (33.66%) and citations by others were 7325 (66.34%). Mean diachronous self-citation rate was 36.16. Citation time lag was zero for 144 (15.52%) papers and one year for 350 (37.72%) papers. Single authored publications (168) have received 456 (4.13%) citations and 1565 multi-authored publications have received 10585 (95.87%) citations. The core citing authors were: J. P. Mittal (695) followed by V. K. Jain (524), H. Mohan (471), T. Mukherjee (307), R. M Iyer (253), H. Pal (251), J. V. Yakhmi (211), A. V. Sapre (174), D. K. Palit (161), N. M. Gupta (128), and S. K. Kulshrestha (116). Citation life cycles of four highly cited papers was discussed. The core journals citing Chemistry Division publications were: J. Phys. Chem.-A (436 citations), Chem. Phys. Lett. (372), J. Phys. Chem. (355), J. Chem. Phys. (353), J. Organomet. Chem. (285), J. Phys. Chem.-B (279), J. Photochem. Photobiol.-A (263), Langmuir (245), J. Am. Chem. Soc. (226), Physica-C (225), Radiat. Phys. Chem. (217), Inorg. Chem. (215) and Indian J. Chem.-A (207).     

On the validity of citation counting in science evaluation: Content analyses of references and citations in psychological publications

In reference to the increasing significance of citation counting in evaluations of scientists and science institutes as well as in science historiography, it is analyzed empirically what is cited in which frequency and what types of citations in scientific texts are used. Content analyses refer to numbers of references, self-references, publication language of references cited, publication types of references cited, and type of citation within the texts. Validity of citation counting is empirically analyzed with reference to random samples of English and German journal articles as well as German textbooks, encyclopedias, and test-manuals from psychology. Results show that 25% of all citations are perfunctory, more than 50% of references are journal articles and up to 40% are books and book-chapters, 10% are self-references. Differences between publications from various psychological sub-disciplines, publication languages, and types of publication are weak. Thus, validity of evaluative citation counting is limited because at least one quarter refers to perfunctory citations exhibiting a very low information utility level and by the fact that existing citation-databases refer to journal articles only.     

Using content-based and bibliometric features for machine learning models to predict citation counts in the biomedical literature

The most popular method for judging the impact of biomedical articles is citation count which is the number of citations received. The most significant limitation of citation count is that it cannot evaluate articles at the time of publication since citations accumulate over time. This work presents computer models that accurately predict citation counts of biomedical publications within a deep horizon of 10 years using only predictive information available at publication time. Our experiments show that it is indeed feasible to accurately predict future citation counts with a mixture of content-based and bibliometric features using machine learning methods. The models pave the way for practical prediction of the long-term impact of publication, and their statistical analysis provides greater insight into citation behavior.     

A scientometric analysis of international LIS journals: Productivity and characteristics

This paper presents a quantitative study of productivity, characteristics and various aspects of global publication in the field of library and information science (LIS). A total of 894 contributions published in 56 LIS journals indexed in SSCI during the years of 2000-2004 were analyzed. A total of 1361 authors had contributed publications during the five years. The overwhelming majority (89.93%) of them wrote one paper. The average number of authors per paper is 1.52. All the studied papers were published in English. The sum of research output of the authors form USA and UK reaches 70% of the total productivity. Most papers received few citations. Each article received on an average 1.6 citations and the LIS researchers cite mostly latest articles. About 48% of citing authors had tendency of self-citation. The productive authors, their contribution and authorship position are listed to indicate their productivity and degree of involvement in their research publications.