The influence of references per paper in the SCI to Impact Factors and the Matthew Effect

All references data was extracted from the annual volumes of the CD-Edition of Science Citation Index (SCI) and the Web of Science of the Institute for Scientific Information (ISI), the journal citation and self-citation data extracted from the Journal Citation Report (JCR), the self-citing rate and self-cited rate calculated based on the JCR method. To determine the trend of mean value of references per paper throughout 1970–2005, a total number of 10,000 records were randomly chosen for each year of under study, and the mean value of references per paper was calculated. To determine the growth of journals IF a total number of 5,499 journals were chosen in the JCR in 2002 and the same set of journals in the year 2004. To show the trend of journals IF, all journals indexed in the JCR throughout 1999–2005 were extracted and the mean values of their IFs was calculated annually. The study showed that the number of references per paper from 1970 to 2005 has steady increased. It reached from 8.40 in 1970 to 34.63 in 2005, an increase of more than 4 times. The majority of publications (76.17%) were in the form of Journals Article. After articles, Meeting Abstracts (9.46%), Notes (3.90%) and Editorial Material (3.78%) are the most frequented publication forms, respectively. 94.57% of all publications were in English. After English, German (1.50%), Russian (1.48%) and French (1.37%) were the most frequented languages, respectively. The study furthermore showed that there is a significant correlation between the IF and total citation of journals in the JCR, and there is an important hidden correlation between IF and the self-citation of journals. This phenomena causes the elevation of journals IF. The more often a journal is citing other journals, the more often it is also cited (by a factor of 1.5) by others. In consequence the growing percentage of journal self-citation is followed by journal self-citedness, which can be considered as the Matthew Effect. There is a linear correlation between journal self-citing and journal self-cited value, the mean value of self-cited rate always stays higher than the self-citing rate. The mean value of self-cited rate in 2000 was 14% and the mean value of self-citing rate is 6.61%, whereas the mean value of self-cited rate in 2005 was 12% and the mean value of self-citing rate was 7.81%.     

On the validity of citation counting in science evaluation: Content analyses of references and citations in psychological publications

In reference to the increasing significance of citation counting in evaluations of scientists and science institutes as well as in science historiography, it is analyzed empirically what is cited in which frequency and what types of citations in scientific texts are used. Content analyses refer to numbers of references, self-references, publication language of references cited, publication types of references cited, and type of citation within the texts. Validity of citation counting is empirically analyzed with reference to random samples of English and German journal articles as well as German textbooks, encyclopedias, and test-manuals from psychology. Results show that 25% of all citations are perfunctory, more than 50% of references are journal articles and up to 40% are books and book-chapters, 10% are self-references. Differences between publications from various psychological sub-disciplines, publication languages, and types of publication are weak. Thus, validity of evaluative citation counting is limited because at least one quarter refers to perfunctory citations exhibiting a very low information utility level and by the fact that existing citation-databases refer to journal articles only.     

Characteristics and impact of grant-funded research: a case study of the library and information science field

This paper reports on a bibliometric study of the characteristics and impact of research in the library and information science (LIS) field which was funded through research grant programs, and compares it with research that received no extra funding. Seven core LIS journals were examined to identify articles published in 1998 that acknowledge research grant funding. The distribution of these articles by various criteria (e.g., topic, affiliation, funding agency) was determined. Their impact as indicated by citation counts during 1998–2008 was evaluated against that of articles without acknowledging extra funding and published in the same journals in the same year using citation data collected from Scopus’ Citation Tracker. The impact of grant-funded research as measured by citation counts was substantially higher than that of other research, both overall and in each journal individually. Scholars from outside LIS core institutions contributed heavily to grant-funded research. The two highest-impact publications by far reported non-grant-based research, and grant-based funding of research reported in core LIS journals was biased towards the information retrieval (IR) area, particularly towards research on IR systems. The percentage of articles reporting grant-funded research was substantially higher in information-oriented journals than in library-focused ones.     

Discussing some basic critique on Journal Impact Factors: revision of earlier comments

In this study the issue of the validity of the argument against the applied length of citation windows in Journal Impact Factors calculations is critically re-analyzed. While previous studies argued against the relatively short citation window of 1–2 years, this study shows that the relative short term citation impact measured in the window underlying the Journal Impact Factor is a good predictor of the citation impact of the journals in the next years to come. Possible exceptions to this observation relate to journals with relatively low numbers of publications, and the citation impact related to publications in the year of publication. The study focuses on five Journal Subject Categories from the science and social sciences, on normal articles published in these journals, in the 2 years 2000 and 2004.     

Effects of disciplines and countries on citation habits. An analysis of empirical papers in behavioural sciences

The theoretical introductions in empirical journal articles have been analyzed looking for factors determining citation habits. Own-country-biases and English-American predominance in citations were not regularly found. Preferred language of the cited publications and absolute citation frequencies were dependent upon both the disciplines and the countries where the journals are published. However, relative citation frequencies (citations related to the length of the text available) have been found to be rather constant across countries (within psychology and psychiatry, respectively) which indicates no such dependence.     

Development, significance and background information about the “Chinese Book Citation Index” (CBkCI) demonstration database

Writing academic books is one of the core expressions of academic research. In the process of writing, the author cites many types of publications such as journals, journal articles, reports, web sources and books. Collecting and analyzing these citations in selected academic books, leads to the creation of book citation indexes. Based on this concept and design, the Chinese book citation index (CBkCI) is produced. The value of the CBkCI lies not only in filling the domestic vacuum in book citation, but also in promoting the quality of academic book publishing, and in contributing to a better library collection development. More importantly it helps to lay a solid foundation in the area of academic evaluation.     

Correlation between impact and collaboration

We obtained data of statistical significance to verify the intuitive impression that collaboration leads to higher impact. We selected eight scientific journals to analyze the correlations between the number of citations and the number of coauthors. For different journals, the single-authored articles always contained the lowest citations. The citations to those articles with fewer than five coauthors are lower than the average citations of the journal. We also provided a simple measurement to the value of authorship with regards to the increase number of citations. Compared to the citation distribution, similar but smaller fluctuations appeared in the coauthor distribution. Around 70% of the citations were accumulated in 30% of the papers, while 60% of the coauthors appeared in 40% of the papers. We find that predicting the citation number from the coauthor number can be more reliable than predicting the coauthor number from the citation number. For both citation distribution and coauthor distribution, the standard deviation is larger than the average value. We caution the use of such an unrepresentative average value. The average value can be biased significantly by extreme minority, and might not reflect the majority.     

The ‘language preference’ in sociology: Measures of ‘language self-citation’, ‘relative own-language preference indicator’, and ‘mutual use of languages’

Although between one-third to one-half of world social sciences research literature is published in languages other than English, studies show very scant use of it by American and English scholars. Almost all studies, however, were conducted from the Anglo-Saxon perspective, limiting the scope of the study to English-published sources or English-speaking scientists and research workers. The present study aimed at assessing the scope of the language preference in a social sciences field, not only among American and British scholars, but among German and French ones as well, using the technique of citation analysis. Samples including mostly 50–60 original research articles were drawn from the 1985–1994 volumes of nine leading sociology journals published in the US, UK, Germany and France and the references appended to each were scrutinized in order to determine the frequency distribution of the languages cited in each periodical. Findings clearly showed a strong preference of writers to cite material in their own language. However, the extent of this bias differed from journal to journal. The American and British writers rank first, with close to 99% of their references being in English. German scholars rank next, preferring German sources in 75% of the cases, and French scholars quote French sources in only 66% of their references. In order to calculate the new refined measure of ‘relative own-language preference’ (ROLP) indicator, the proportions of ‘language self-citation’ were related to the estimated proportions of these languages in the existing body of sociology research. This measure reveals that German sociologists have the strongest bias towards their mother-tongue, their ratio of references in German exceeding almost 12 to 28 times the expected figure according to the German language share in sociology research. Next come French sociologists (8 to 14 times) while American and British ones display the lowest own-language bias, only slightly higher than expected. Further analysis of the foreign languages preference of each group, according to a ‘mutual-use’ matrix, shows a relative low use of German and French sources by British-American sociologists.     

A comparative study of interdisciplinary changes between information science and library science

This study employs the method of direct citation to analyze and compare the interdisciplinary characteristics of the two disciplines of library science and information science during the period of 1978–2007. Based on the research generated by five library science journals and five information science journals, library science researchers tend to cite publications from library and information science (LIS), education, business/management, sociology, and psychology, while researchers of information science tend to cite more publications from LIS, general science, computer science, technology, and medicine. This means that the disciplines with larger contributions to library science are almost entirely different from those contributing to information science. In addition, researchers of library science frequently cite publications from LIS; the rate is as high as 65.61%, which is much higher than the rate for information science, 49.50%. However, a decreasing trend in the percentage of LIS in library science indicates that library science researchers tend to cite more publications from non-LIS disciplines. A rising trend in the proportion of references to education sources is reported for library science articles, while a rising trend in the proportion of references to computer science sources has been found for information science articles. In addition, this study applies an interdisciplinary indicator, Brillouin’s Index, to measurement of the degree of interdisciplinarity. The results confirm that the trend toward interdisciplinarity in both information science and library science has risen over the years, although the degree of interdisciplinarity in information science is higher than that in library science.     

Bibliometric analysis of biological invasions research during the period of 1991 to 2007

The objective of this study is to conduct a bibliometric analysis of all biological invasions-related publications in the Science Citation Index (SCI) from 1991 to 2007. The indicator citation per publication (CPP) was used to evaluate the impact of articles, journals, and institutions. In the 3323 articles published in 521 journals, 7261 authors from 1905 institutions of 100 countries participated. As the most productive country of biological invasions research, the US will benefit from more collaboration between institutions, countries, and continents. In addition, analysis of keywords was applied to reveal research trends.     

The Garfield impact factor, one of the fundamental indicators in scientometrics

The paper summarizes some basic features of the Garfield impact factor (GF). Accordingly, GF should be regarded as a scientometric indicator representing the relative contribution of journals to the total impact of information in a field. For calculating GF, both from theoretical and practical reasons the “ratio of the sums” method is recommended over the “mean of the ratios” method. Scientific advances are made by the most influential, presumably most frequently cited articles. The distribution of citations among the publications is skewed in journals. Consequently, the GF index will be influenced primarily by the highly cited papers. It follows, GF represents the most valuable part of the information in journals quantitatively, and even therefore it may be regarded as a reliable impact indicator.     

Assessing non-standard article impact using F1000 labels

Faculty of 1000 (F1000) is a post-publishing peer review web site where experts evaluate and rate biomedical publications. F1000 reviewers also assign labels to each paper from a standard list or article types. This research examines the relationship between article types, citation counts and F1000 article factors (FFa). For this purpose, a random sample of F1000 medical articles from the years 2007 and 2008 were studied. In seven out of the nine cases, there were no significant differences between the article types in terms of citation counts and FFa scores. Nevertheless, citation counts and FFa scores were significantly different for two article types: “New finding” and “Changes clinical practice”: FFa scores value the appropriateness of medical research for clinical practice and “New finding” articles are more highly cited. It seems that highlighting key features of medical articles alongside ratings by Faculty members of F1000 could help to reveal the hidden value of some medical papers.     

Impact of open access on citation of scholarly publications in the field of civil engineering

The development of science is accompanied by growth of scholarly publications, primarily in the form of articles in peer-reviewed journals. Scientific work is often evaluated through the number of scientific publications in international journals and their citations. This article discusses the impact of open access (OA) on the number of citations for an institution from the field of civil engineering. We analyzed articles, published in 2007 in 14 international journals with impact factor, which are included in the Journal Citation Reports subject category “Civil Engineering”. The influence of open access on the number of citations was analyzed. The aim of our research was to determine if open access articles from the field of civil engineering receive more citations than non-open access articles. Based on the value of impact factor and ranking in quartiles, we also looked at the influence of the rank of journals on the number of citations, separately for OA and Non OA articles, in databases Web of Science (WOS), Scopus and Google Scholar. For 2,026 studied articles we found out that 22 % of them were published as OA articles. They received 29 % of all citations in the observed period. We can conclude by the significance level 5 % or less that in the databases WOS and Scopus the articles from top ranked journals (first quartile) achieved more citations than Non OA articles. This argument can be confirmed for some other journals from second quartile as well, while for the journals ranked into the third quartile it can’t be confirmed. This could be confirmed only partly for journals from the second quartile, and would not be confirmed for journals ranked into the third quartile. This shows that open access is not a sufficient condition for citation, but increases the number of citations for articles published in journals with high impact.     

Do lead articles signal higher quality in the digital age? Evidence from finance journals

Citations are regarded as measures of quality yet citation rates vary widely within each of the top finance journals. Since article ordering is at the discretion of editors, lead articles can be interpreted as signals of quality that academics can use to allocate their attention and assert the value of their publications. Advances in electronic journal access allow researchers to directly access articles, suggesting article ordering may be less relevant today. We confirm the past importance of lead articles by examining citation rates from published papers as well as the wider source of papers that are listed in Google Scholar. Our findings also confirm using Google Scholar as a citation source provides congruent results to using citations from articles published in ISI-listed journals, with the additional benefit of it potentially being more timely since it includes wider citation sources, inclusive of working and conference papers.     

Publication types,citation rates and evaluation

In order to resolve questions frequently raised in the context of research evaluation about the citation rates of journal publications in relation to other types of publications, the total research output of substantial institutions or systems has to be brought under bibliographic control. That precondition has rarely been met: there are few published studies of the total range of publications of major research institutions, including books, book chapters, technical reports and published conference proceedings. The Research Evaluation and Policy Project (REPP) at the Australian National University (ANU) has established a database covering all the publications from the Institute of Advanced Studies (IAS), a fulltime research institution at the ANU, and has examined in detail citations in the journal literature accruing to all types of publications. The database contains a significant number of publications, nearly 30 000 items, and covers the sciences and the social sciences and humanities. This data enables us to examine whether the citation record of research publications appearing in journals indexed by the Institute for Scientific Information (ISI) is a useable surrogate for the citation record within ISI journals of all model of publication. We contend that, if certain preconditions are met, the choice of citation rate is not critical.     

How does scientific success relate to individual and organizational characteristics? A scientometric study of psychology researchers in the German-speaking countries

Purpose: To provide up-to-date bibliometric reference data describing the output and success of psychology researchers in the German-speaking countries, including lifetime publication and citation numbers, and to investigate associations of bibliometric measures with academic status and gender as well as the department characteristics of size and quota of senior researchers. Method Queried literature databases using an extensive online register of academic psychologists in the German-speaking countries, obtaining valid data for 85 % (N = 1742) of the population of interest. Findings Skewed distributions for publications and citations; maximum number of German-language (=native) publications much higher than maximum number of English-language publications; relatively large part of population publishing almost exclusively in German; publication count predictable by academic status, gender, department size, and quota of senior researchers; citation count predictable by publication count, status, department size, and quota of senior researchers; department characteristics interact with individual characteristics to produce specific conditions under which publication count and citation count are higher or lower than expected: combination of female gender, small department size and large quota of senior researchers is associated with particularly increased publication count; female gender and large department size are associated with decreased publication count; high publication count, large department size and low quota of senior researchers are associated with increased citation count; low publication count and large quota of senior researchers are associated with decreased citation count. Conclusions Reference values for scientific output provided in this study provide an anchor for monitoring and international comparison; despite considerable noise in data, we show that interactions of individual and organizational characteristics are relevant for scientific success and should be investigated further, e.g. by adopting various measures of organizational diversity and tracing a population longitudinally.     

Quantitative study on Australian academic science

As at the end of 2009, 50 million scholarly papers were in existence, and changes in the global higher education sector are seeing this number grow rapidly. With the impact of university ranking schemes, dramatic changes have occurred in the academic publication system; both the number of publications, and the number of scholars, are increasing rapidly. In this research, we conduct a focused study on the quantitative relationships in Australian academic science. Sample data is selected from three institutions, representing three types of universities in Australia: a leading university, a middle-tier university, and a non-comprehensive university. For our analysis, bibliometric data for 32,056 scholars and 353,334 publications are downloaded. The results show that varying growth rates for scholars (5.6, 6.1, and 7.7%) and journal publications (6.7, 7.4, and 9.7%) for the leading university, the middle-tier university, and the non-comprehensive university respectively. We also evaluate how scholarly publications differ across these three Australian universities types from six aspects: average number of publications per scholar, percentage of first-authored publications, collaboration pattern, average SJR score of publishing journals, percentage of Quartile category of publishing journals, and average citation per publication. Lastly, we consider how the scholarship life-cycle varies in Australian universities in terms of publishing age, thus establishing the quantitative relationship between scholarship life-cycle and publishing performance in Australian universities.     

Validation of bibliometric indicators in the field of mathematics

Bibliometric analyses of scientific publications provide quantitative information that enables evaluators to obtain a useful picture of a team's research visibility. In combination with peer judgements and other qualitative background knowledge, these analyses can serve as a basis for discussions about research performance quality. However, many mathematicians are not convinced that citation counts do in fact provide useful information in the field of mathematics. According to these mathematicians, citation and publication habits differ completely from scholarly fields such as chemistry or physics. Therefore, it is impossible to derive valid information regarding research performance from citation counts. The aim of this study is to obtain more insight into the significance of citation-based indicators in the field of mathematics. To which extent do citation-scores mirror to the opinions of experts concerning the quality of a paper or a journal? A survey was conducted to answer this question.Top journals, as qualified by experts, receive significantly higher citation rates thangood journals. Thesegood journals, in turn, have significantly higher scores than journals with the qualificationless good. Top publications, recorded in the ISI database, receive on the average 15 times more citations than the mean score within the field of mathematics as a whole. In conclusion, the experts' views on top publications or top journals correspond very well to bibliometric indicators based on citation counts.     

Reconsidering the gold open access citation advantage postulate in a multidisciplinary context: an analysis of the subject categories in the Web of Science database 2009–2014

Since Lawrence in 2001 proposed the open access (OA) citation advantage, the potential benefit of OA in relation to citation impact has been discussed in depth. The methodology to test this postulate ranges from comparing the impact factors of OA journals versus traditional ones, to comparing citations of OA versus non-OA articles published in the same non-OA journals. However, conclusions are not entirely consistent among fields, and two possible explications have been suggested in those fields where a citation advantage has been observed for OA: the early view and the selection bias postulates. In this study, a longitudinal and multidisciplinary analysis of gold OA citation advantage is developed. All research articles in all journals for all subject categories in the multidisciplinary database Web of Science are considered. A total of 1,138,392 articles—60,566 (5.3%) OA articles and 1,077,826 (94.7%) non-OA articles—published in 2009 are analysed. The citation window considered goes from 2009 to 2014, and data are aggregated for the 249 disciplines (subject categories). At journal level, we also study the evolution of journal impact factors for OA and non-OA journals in those disciplines whose OA prevalence is higher (top 36 subject categories). As the main conclusion, there is no generalizable gold OA citation advantage, neither at article nor at journal level.