Towards a new crown indicator: an empirical analysis

We present an empirical comparison between two normalization mechanisms for citation-based indicators of research performance. These mechanisms aim to normalize citation counts for the field and the year in which a publication was published. One mechanism is applied in the current so-called crown indicator of our institute. The other mechanism is applied in the new crown indicator that our institute is currently exploring. We find that at high aggregation levels, such as at the level of large research institutions or at the level of countries, the differences between the two mechanisms are very small. At lower aggregation levels, such as at the level of research groups or at the level of journals, the differences between the two mechanisms are somewhat larger. We pay special attention to the way in which recent publications are handled. These publications typically have very low citation counts and should therefore be handled with special care.     

A study on the citation situation within the citing paper: citation distribution of references according to mention frequency

Citation impact indicators play a significant role in evaluating the scientific research activity. Most of citation impact indicators are based on the citation count that the publication is cited as a reference in the other publications, but the difference between each citation situation was not considered. Normally, the number of citations that a publication is cited in the other publications may represent the formal quality of the publication. Similarly, the number of times that a publication is really mentioned within the citing publication, it may also represent the formal quality of the citation. We have examined about how many times each reference was really mentioned within the citing publications and studied about the citation situation within the citing publications. We verified that the citation distribution of references according to the mention frequency follows the Generalized Pareto distribution. The results showed that about 20% of total references were mentioned three and more times, and the number of citation mentions for the about 50% of total references were from about 20% of the total references in the citing publications.     

Citation regression analysis of computer science publications in different ranking categories and subfields

A number of bibliometric studies point out that the role of conference publications in computer science differs from that in other traditional fields. Thus, it is interesting to identify the relative status of journal and conference publications in different subfields of computer science based on the citation rates categorised by the China Computer Federation (CCF) classifications and venue types. In this research, we construct a dataset containing over 100,000 papers recommended by the CCF catalogue and their citation information. We also investigate some other factors that often influence a paper’s citation rate. An experimental study shows that the relative status of journals and conferences varies greatly in different subfields of computer science, and the impact of different publication levels varies according to the citation rate. We also verify that the classification of a publication, number of authors, maximum h-index of all authors of a paper, and average number of papers published by a publication have different effects on the citation rate, although the citation rate may have a different degree of correlation with these factors.     

Do interdisciplinary research teams deliver higher gains to science?

The present paper takes its place in the stream of studies that analyze the effect of interdisciplinarity on the impact of research output. Unlike previous studies, in this study the interdisciplinarity of the publications is not inferred through their citing or cited references, but rather by identifying the authors’ designated fields of research. For this we draw on the scientific classification of Italian academics, and their publications as indexed in the WoS over a 5-year period (2004–2008). We divide the publications in three subsets on the basis the nature of co-authorship: those papers coauthored with academics from different fields, which show high intensity of inter-field collaboration (“specific” collaboration, occurring in 110 pairings of fields); those papers coauthored with academics who are simply from different “non-specific” fields; and finally co-authorships within a single field. We then compare the citations of the papers and the impact factor of the publishing journals between the three subsets. The results show significant differences, generally in favor of the interdisciplinary authorships, in only one third (or slightly more) of the cases. The analysis provides the value of the median differences for each pair of publication subsets.     

Use of citation per publication as an indicator to evaluate pentachlorophenol research

The objective of the study was to perform a bibliometric analysis of all pentachlorophenolrelated publications in the Science Citation Index (SCI). Analyzed parameters included document type, language of publication, page count, publication output, authorship, keywords plus, publication pattern, citation and country of publication. The US produced 29% of the total single country publications where the seven major industrial countries accounted for the majority of the total production (66%). An indicator citation per publication was successfully applied in this study to evaluate the impact of number of authors, countries, and journals. The mean value of citation per publication of collaborative papers was higher than that of single country publications. In addition analysis of keywords plus in different period was applied to indicate a research trend.     

Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines

For practical reasons, bibliographic databases can only contain a subset of the scientific literature. The ISI citation databases are designed to cover the highest impact scientific research journals as well as a few other sources chosen by the Institute for Scientific Information (ISI). Google Scholar also contains citation information, but includes a less quality controlled collection of publications from different types of web documents. We define Google Scholar unique citations as those retrieved by Google Scholar which are not in the ISI database. We took a sample of 882 articles from 39 open access ISI-indexed journals in 2001 from biology, chemistry, physics and computing and classified the type, language, publication year and accessibility of the Google Scholar unique citing sources. The majority of Google Scholar unique citations (70%) were from full-text sources and there were large disciplinary differences between types of citing documents, suggesting that a wide range of non-ISI citing sources, especially from non-journal documents, are accessible by Google Scholar. This might be considered to be an advantage of Google Scholar, since it could be useful for citation tracking in a wider range of open access scholarly documents and to give a broader type of citation impact. An important corollary from our study is that Google Scholar’s wider coverage of Open Access (OA) web documents is likely to give a boost to the impact of OA research and the OA movement.     

Extending citer analysis to journal impact evaluation

The concept of citer analysis investigated earlier by Ajiferuke and Wolfram (In: B. Larsen, J. Leta (eds.) Proceedings of the 12th international conference of the international society for scientometrics and informetrics (ISSI) pp. 798–808, 2009, Scientometrics 83:623–638, 2010) is extended to journals where different citing units (citers, citing articles, citing journals) are compared with the journal impact factor and each other to determine if differences in ranking arise from different measures. The citer measures for the 31 high impact journals studied from information science and library science are significantly correlated, even more so than the earlier citer analysis findings, indicating that there is a close relationship among the different units of measure. Still, notable differences in rankings for the journals examined were evident for the different measures used, especially from either 5-year impact factor or number of citing articles per publication to the number of citing journals per publication. The journals that are adversely affected seem to be those whose citations are concentrated in a few journals. This informed the need to develop a journal citation concentration index, which can serve as a complementary measure to the existing journal impact indices.     

Renewable energy research 1995–2009: a case study of wind power research in EU, Spain, Germany and Denmark

The paper reports the developments and citation patterns over three time periods of research on Renewable Energy generation and Wind Power 1995–2011 in EU, Spain, Germany and Denmark. Analyses are based on Web of Science and incorporate journal articles as well as conference proceeding papers. Scientometric indicators include publication collaboration ratios, top-player distribution as well as citedness and correspondence analyses of citing publications, relative citation impact, distributions of top-cited as well as top-citing institutions and publication sources and cluster analysis of citing title terms to map knowledge export areas. Findings show an increase in citation impact for Renewable Energy and Wind Power research albeit hampered by scarcely cited conference papers. Although EU maintains its global top position in producing Renewable Energy and Wind Power research the developments of EU and German world shares as well as citation impact are negative during the most recent 7 year period. During the same time the citation impact of Spain and Denmark increase and place both nations among the top-ranking countries in Wind Power research. Spain is the only EU country that increases its world production share from 2000. China is currently ranked three after EU and USA in research output, however with a very low citation impact. Spain, Denmark and Germany each demonstrates distinct collaboration patterns and publication source and citation distribution profiles. More than half the citations to EU Wind Power research are EU-self citations. An expected intensified EU collaboration in the Wind Energy field does not come about. The most productive research institutions in Denmark and Spain are also the most cited ones.     

Journals of general &; internal medicine and surgery: An analysis and comparison of citation

Summary The purpose of this study is to analyze and compare journal citation data, from Journal Citation Reports on the Web 2000, of general and internal medicine and surgery. The source items and five kinds of citation data, i.e. citation counts, impact factor, immediacy index, citing half-life and cited half-life are examined and the correlation between each of the fifteen pairs of citation data is determined based on the Pearson correlation tests. The Fishers Z-transform was employed to test the significant difference between the Pearson correlation coefficient for each pair of citation data of these two subject areas. The following results of this work reveal: the frequently published journals are cited more frequently and also with high impact factor and immediacy index, in addition, they are usually accompanied with short citing half-life (i.e., usually cite current literature). The impact factor and immediacy index has significant correlation with citation counts. A significant correlation also exists between impact factor and immediacy index. However there is no correlation between cited half-life and other citation data, except citing half-life. For journals of general and internal medicine and surgical medicine, there are no significant difference of the Pearson correlation coefficient for the following pair of citation data: source items and citation counts, source items and impact factor, source items and citing half-life, citation counts and citing half-life, impact factor and citing half-life, immediacy index and citing half-life, and cited half-life and citing half-life.     

Internationalization of scientific journals: A measurement based on publication and citation scope

Although impact factor and related measurements are the best-known features of scientific journals, other characteristics are of particular interest. The way a journal reflects the internationalized nature of science may be determined by many methods, one of which is based on the distribution of authoring and citing countries. This can be systematically measured either by a comparison of these distributions with averages profiles of a discipline or specialty, or by concentration indexes on the other. This paper focuses on the first approach. As the average profile of science drifts with the level of visibility, stratification by impact level is discussed. In this study, experimental internationalization indexes were calculated on the SCI for journals belonging to Earth&Space and Applied Biology. Convergence of measurements (types of indexes, type of normalization, publication vs citation scope) is adressed. Internationalization indexes may have a variety of applications, including characterization of the scientific publishing market and sampling of the SCI for science indicators.     

The nature and relationship between the productivity of journals and their citations in semiconductor literature

The purpose of this study was to investigate the relationship between journals" productivity and their citations in the field of semiconductors. Journal samples were gathered from the INSPEC database, 1978 to 1997 while the data of citation frequency, impact factor, cited half-life and citing half-life were obtained from Science Citation Index, Journal Citation Reports 1997 CD-ROM edition. One thousand and eight hundred and seventy seven journals publishing articles on semiconductors were retrieved. The nature for the data of journal productivity, impact factor, cited half-life and citing half-life are explored. Among these journals, only 672 journals that were covered in JCR were compared. Moreover, statistical tests of more productive journals with cumulative publication in semiconductor >100 were also conducted on the basis of all articles they published annually (for 1997). The results of the study showed that there is a significant correlation between journal productivity and citation frequency and between journal productivity and impact factor. However, there are no associations between journal productivity and cited half-life and between journal productivity and citing half-life. This revised version was published online in August 2006 with corrections to the Cover Date.     

The influence of publication delays on impact factors

Summary Based on the convolution formula of the disturbed aging distribution (Egghe&Rousseau, 2000) and the transfer function model of the publishing delay process, we establish the transfer function model ofthe disturbed citing process. Using the model, we make simulative investigations of disturbed citation distributions and impact factors according to different average publication delays. These simulative results show that the bigger increment the average publication delays in a scientific field, the larger shift backwards of the citation distribution curves and the more fall the impact factors of journals in the field. Based on sometheoretical hypotheses, it is shown that there exists theoretically an approximate inverse linear relation between the field (or discipline) average publication delay and the journal impact factor.     

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

     

Interdisciplinarity and the intellectual base of literature studies: citation analysis of highly cited monographs

This article studies interdisciplinarity and the intellectual base of 34 literature journals using citation data from Web of Science. Data from two time periods, 1978–1987 and 1998–2007 were compared to reveal changes in the interdisciplinary citing of monographs. The study extends the analysis to non-source publications; using the classification of monographs to show changes in the intellectual base. There is support for increased interdisciplinary citing of sources, especially to the social sciences, and changes in the intellectual base reflect this. The results are explained using theories on the intellectual and social organization of scientific fields and the use of bibliometric methods on the humanities is discussed. The article demonstrates how citation analysis can provide insights into the communication patterns and intellectual structure of scholarly fields in the arts and humanities.     

Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles

Context. The use of citation frequency and impact factor as measures of research quality and journal prestige is being criticized. Citation frequency is augmented by self-citation and for most journals the majority of citations originate from a minority of papers. We hypothesized that citation frequency is also associated with the geographical origin of the research publication. Objective. We determined whether citations originate more frequently from institutes that are located in the same country as the authors of the cited publication than would be expected by chance. Design. We screened citations referring to 1200 cardiovascular publications in the 7 years following their publication. For the 1200 citation recipient publications we documented the country where the research originated (9 countries/regions) and the total number of received citations. For a selection of 8864 citation donor papers we registered the country/region where the citing paper originated. Results. Self-citation was common in cardiovascular journals (n = 1534, 17.8%). After exclusion of self-citation, however, the number of citations that originated from the same country as the author of the citation recipient was found to be on average 31.6% higher than would be expected by chance (p<0.01 for all countries/regions). In absolute numbers, nation oriented citation bias was most pronounced in the USA, the country with the largest research output (p<0.001). Conclusion. Citation frequency was significantly augmented by nation oriented citation bias. This nation oriented citation behaviour seems to mainly influence the cumulative citation number for papers originating from the countries with a larger research output.     

Validation of bibliometric indicators in the field of mathematics

Bibliometric analyses of scientific publications provide quantitative information that enables evaluators to obtain a useful picture of a team's research visibility. In combination with peer judgements and other qualitative background knowledge, these analyses can serve as a basis for discussions about research performance quality. However, many mathematicians are not convinced that citation counts do in fact provide useful information in the field of mathematics. According to these mathematicians, citation and publication habits differ completely from scholarly fields such as chemistry or physics. Therefore, it is impossible to derive valid information regarding research performance from citation counts. The aim of this study is to obtain more insight into the significance of citation-based indicators in the field of mathematics. To which extent do citation-scores mirror to the opinions of experts concerning the quality of a paper or a journal? A survey was conducted to answer this question.Top journals, as qualified by experts, receive significantly higher citation rates thangood journals. Thesegood journals, in turn, have significantly higher scores than journals with the qualificationless good. Top publications, recorded in the ISI database, receive on the average 15 times more citations than the mean score within the field of mathematics as a whole. In conclusion, the experts' views on top publications or top journals correspond very well to bibliometric indicators based on citation counts.     

Creation of journal-based publication profiles of scientific institutions — A methodology for the interdisciplinary comparison of scientific research based on the J-factor

A form of normalisation is presented for the evaluation of citation data on multidisciplinary research. This method is based on the existing classification according to the publishing journals and not on the classification of output according to ISI subject categories. A publication profile is created for each institution to be investigated. This profile accounts for the weight of publications in a journal, represented by the number of publications as a proportion of the total output of the institution. In accordance with this weight, the citation rate of each journal is compared to a qualified relative indicator. The final result is a relative citation rate J, which is the relative perception of the performance of an institution accounting for its publication and citation habits and makes a transdisciplinary comparison possible.     

Publication activity, citation impact and bi-directional links between publications and patents in biotechnology

The study focuses on publication activity, citation impact and citation links between publications and patents in biotechnology. The European Union (EU), US, Japan and China are the most important global players. However, the landscape is changing since the EU and the US are losing ground because of challenges from a group of emerging economies. National profiles differ between the two groups of main players and upcoming countries; the focus on red biotechnology in the US and Europe is contrasted by propensity for white and green technology in Asia. Furthermore, the subject profile of biotechnology papers citing patents and cited by patents as well as the relationship between patent citations and citation impact in scientific literature is explored. Papers that cite patents tend to reflect propensity towards white biotechnology while patent-cited publications have a higher relative share in red biotechnology. No significant difference concerning the citation impact of publications ??citing patents?? and ??not citing patents?? can be found. This is contrasted by the observation that patent-cited papers perform distinctly better in terms of standard bibliometric indicators than comparable publications that are not linked to technology in this direction.     

The influence of the publication delay on journal rankings according to the impact factor

Summary  The inter-citation journal group is defined as a group of journals with inter-citation relations. In this paper, according to the 2003 JCR, an inter-citation relation matrix of 10 medical journals is established. Based on the transfer function model of the disturbed citing process, the calculation formula of journal impact factor disturbed by publication delays of certain journal in the group is deduced and a changing process of every journal's impact factor caused by the increase of each journal's average publication delay is simulated. In the inter-citation journal group, when a journal's publication delay increase, impact factors of all journals will be decreased and rankings of journals according to the impact factor may be changed. The closer a citation relation between two journals, the stronger the interaction of them and the larger the decrease of their impact factors caused by the increase of their publication delays.     

Research performance indicators for university departments: A study of an agricultural university

The present bibliometric study extends previous work by focusing on the research performance of departments in the natural and life sciences, the social and behavioral sciences, and the humanities. The present study covers all 70 departments from one agricultural university, and several veterinary departments of a second university. The impact analysis was extended by including other types of documents than journal articles. For about a third of the departments, publications not covered in citation indexes accounted for at least 30% of the citations to their total oeuvre. To deal with different citation and publication habits in the various fields, both short-term and medium-term impact assessments were made. The commonly used three year window is not universally applicable, as our results show. The inclusion of self-citations forms an important source of error in the ratio of actual/expected impact. To cope with this, the trend and level of self-citations was compared at university level with that in a matched sample of publications. Moreover, at a departmental level, self-citation rates were used to detect departments with divergent levels of self-citation. The expected impact of journals accounted for only 18% of the variance in actual impact. Comparison of bibliometric indicators with two peer evaluations showed that the bibliometric impact analyses provided important additional information.