Effect on the journal impact factor of the number and document type of citing records: a wide-scale study

We studied the effect on journal impact factors (JIF) of citations from documents labeled as articles and reviews (usually peer reviewed) versus citations coming from other documents. In addition, we studied the effect on JIF of the number of citing records. This number is usually different from the number of citations. We selected a set of 700 journals indexed in the SCI section of JCR that receive a low number of citations. The reason for this choice is that in these instances some citations may have a greater impact on the JIF than in more highly-cited journals. After excluding some journals for different reasons, our sample consisted of 674 journals. We obtained data on citations that contributed to the JIF for the years 1998?C2006. In general, we found that most journals obtained citations that contribute to the impact factor from documents labeled as articles and reviews. In addition, in most of journals the ratio between citations that contributed to the impact factor and citing records was greater than 80% in all years. Thus, in general, we did not find evidence that citations that contributed to the impact factor were dependent on non-peer reviewed documents or only a few citing records.     

A citation-analysis of economic research institutes

The citation analysis of the research output of the German economic research institutes presented here is based on publications in peer-reviewed journals listed in the Social Science Citation Index for the 2000–2009 period. The novel feature of the paper is that a count data model quantifies the determinants of citation success and simulates their citation potential. Among the determinants of the number of cites the quality of the publication outlet exhibits a strong positive effect. The same effect has the number of the published pages, but journals with size limits also yield more cites. Field journals get less citations in comparison to general journals. Controlling for journal quality, the number of co-authors of a paper has no effect, but it is positive when co-authors are located outside the own institution. We find that the potential citations predicted by our best model lead to different rankings across the institutes than current citations indicating structural change.     

Crossing the hurdle: the determinants of individual scientific performance

An original cross-sectional dataset referring to a medium-sized Italian university is implemented in order to analyze the determinants of scientific research production at individual level. The dataset includes 942 permanent researchers of various scientific sectors for a 3-year time-span (2008–2010). Three different indicators—based on the number of publications and/or citations—are considered as response variables. The corresponding distributions are highly skewed and display an excess of zero-valued observations. In this setting, the goodness-of-fit of several Poisson mixture regression models are explored by assuming an extensive set of explanatory variables. As to the personal observable characteristics of the researchers, the results emphasize the age effect and the gender productivity gap—as previously documented by existing studies. Analogously, the analysis confirms that productivity is strongly affected by the publication and citation practices adopted in different scientific disciplines. The empirical evidence on the connection between teaching and research activities suggests that no univocal substitution or complementarity thesis can be claimed: a major teaching load does not affect the odds to be a non-active researcher and does not significantly reduce the number of publications for active researchers. In addition, new evidence emerges on the effect of researchers administrative tasks—which seem to be negatively related with researcher’s productivity—and on the composition of departments. Researchers’ productivity is apparently enhanced by operating in department filled with more administrative and technical staff, and it is not significantly affected by the composition of the department in terms of senior/junior researchers.     

Any publicity is better than none: newspaper coverage increases citations, in the UK more than in Italy

A citation advantage for research covered by the mass media is a plausible, but poorly studied phenomenon. Two previous studies, both conducted in the United States, found a positive correlation between media reporting and citations. Only one of these studies was able to conclude that the correlation was caused by a real “publicity effect” rather than by the media highlighting papers that are intrinsically destined to have greater scientific impact (called the ‘earmark’ hypothesis). This study assessed the relative importance of the publicity effect outside the US, by comparing studies published in 2008 and 2009 in the Proceedings of the National Academy of Sciences that had been featured in newspapers in Italy and the United Kingdom. Newspapers in the two countries covered a similar range of topics, and tended to over-represent local (national) research. Compared to studies not appearing in any of the newspapers considered, those featured in British newspapers had around 63 % more citations, whilst in Italian newspapers 16 %. The proportion of citations from Italian authors, however, was significantly increased by newspapers, particularly by those in Italian. The equivalent effect on citations from the UK was smaller and only marginally significant. Studies accompanied by a press release did not receive, overall, significantly more citations. In sum, results suggest that the publicity effect is strongest for English-speaking media, whilst non-English reporting has mostly a local influence. These effects might represent a confounding factor in citation-based research assessment and might contribute to the many biases known to affect the scientific literature.     

Impact of open access on citation of scholarly publications in the field of civil engineering

The development of science is accompanied by growth of scholarly publications, primarily in the form of articles in peer-reviewed journals. Scientific work is often evaluated through the number of scientific publications in international journals and their citations. This article discusses the impact of open access (OA) on the number of citations for an institution from the field of civil engineering. We analyzed articles, published in 2007 in 14 international journals with impact factor, which are included in the Journal Citation Reports subject category “Civil Engineering”. The influence of open access on the number of citations was analyzed. The aim of our research was to determine if open access articles from the field of civil engineering receive more citations than non-open access articles. Based on the value of impact factor and ranking in quartiles, we also looked at the influence of the rank of journals on the number of citations, separately for OA and Non OA articles, in databases Web of Science (WOS), Scopus and Google Scholar. For 2,026 studied articles we found out that 22 % of them were published as OA articles. They received 29 % of all citations in the observed period. We can conclude by the significance level 5 % or less that in the databases WOS and Scopus the articles from top ranked journals (first quartile) achieved more citations than Non OA articles. This argument can be confirmed for some other journals from second quartile as well, while for the journals ranked into the third quartile it can’t be confirmed. This could be confirmed only partly for journals from the second quartile, and would not be confirmed for journals ranked into the third quartile. This shows that open access is not a sufficient condition for citation, but increases the number of citations for articles published in journals with high impact.     

Use of bibliometric information to assist research policy making. A comparison of publication and citation profiles of Full and Associate Professors at a School of Chemistry in Uruguay

Summary Publication and citation profiles of Full and Associate Professors at the School of Chemistry of the Universidad de la República in Uruguay were investigated. The groups do not exhibit markedly different age averages. However, the average time since they started publishing, as well as other characteristics of their publication records, like productivity or citations, set them apart. From the point of view of both the number of papers per author and per year of activity, on one side, and of the number of citations per year of activity, on the other, the group of Full Professors has statistically significant larger averages than the Associate Professors. The impact of self-citations, multi-authorship and internationalization of the publications were analyzed within the two groups and shown to have no excessive or predictable influence on those parameters, except in the case of few (≤ 2) or many (>8) authors. It is suggested in this paper that these two indicators, number of papers per author per production year and number of citations per production year, combined in a plot allowing a bidimensional ranking of the individuals in the groups, may be used profitably as one of the components in the development of a policy toward promotion of Associate Professors. The analysis showed also that the quotient of citations received to number of papers published, even when derived from actual citation data of the scientists without involving the impact factors of the journals in which they publish, are not good parameters to use for that purpose, essentially because there is a reduction in the information content of the indicator with respect to those described before.     

Age effects in scientific productivity

Age effects in scientific production are a consolidated stylised fact in the literature. At the level of scientist productivity declines with age following a predictable pattern. The problem of the impact of age structure on scientific productivity at the level of institutes is much less explored. The paper examines evidence from the Italian National Research Council. The path of hiring of junior researchers along the history of the institution is reconstructed. We find that age structure has a depressing effect on productivity and derive policy implications. The dynamics of growth of research institutes is introduced as a promising research field. This revised version was published online in August 2006 with corrections to the Cover Date.     

Relationship among research collaboration, number of documents and number of citations: a case study in Spanish computer science production in 2000–2009

This paper analyzes the relationship among research collaboration, number of documents and number of citations of computer science research activity. It analyzes the number of documents and citations and how they vary by number of authors. They are also analyzed (according to author set cardinality) under different circumstances, that is, when documents are written in different types of collaboration, when documents are published in different document types, when documents are published in different computer science subdisciplines, and, finally, when documents are published by journals with different impact factor quartiles. To investigate the above relationships, this paper analyzes the publications listed in the Web of Science and produced by active Spanish university professors between 2000 and 2009, working in the computer science field. Analyzing all documents, we show that the highest percentage of documents are published by three authors, whereas single-authored documents account for the lowest percentage. By number of citations, there is no positive association between the author cardinality and citation impact. Statistical tests show that documents written by two authors receive more citations per document and year than documents published by more authors. In contrast, results do not show statistically significant differences between documents published by two authors and one author. The research findings suggest that international collaboration results on average in publications with higher citation rates than national and institutional collaborations. We also find differences regarding citation rates between journals and conferences, across different computer science subdisciplines and journal quartiles as expected. Finally, our impression is that the collaborative level (number of authors per document) will increase in the coming years, and documents published by three or four authors will be the trend in computer science literature.     

A bibliometric analysis of physics publications in Korea, 1994-1998

This study examined research performance of Korean physicists, comparing Korean-authoredpapers versus internationally co-authored papers, indexed in SCI, 1994-1998, and using thenumber of citations received by internationally co-authored papers covered by the SCI CD-ROM.For the study, 4,665 papers published from the researchers affiliated with the physics departmentsor physics-associated laboratories at Korean universities and indexed by SCI were analyzed.Korean authored papers tended to be published in Korean, Japanese, and UK journals, whileinternationally co-authored papers were more likely to appear in German, Dutch, and Swissjournals. Among the 18 authorship countries (on the basis of first author), 93 internationally co-authored papers by U.S. researchers had the highest citation rate, an average 15.9 citations perpaper. Of the eight countries that published over 5 papers, there was no correlation between theaverage number of citations per paper and the total number of citations. However, an ANOVAindicated a significant difference between the average number of citations per paper according tocountry (F = 5.84, p < 0.0005). In other words, papers by the U.S. and French researchers tendedto be cited more frequently than papers by the Italian, Japanese, Korean, Russian, and Germanresearchers.     

A bibliometric analysis of publications by the School of Biological Sciences, Seoul National University, South Korea

This study presents an in-depth survey of research and citation performance of the School of Biological Sciences (SBS) 39-member faculty at Seoul National University (SNU), the most prestigious university in South Korea, for the years 2004–2009. Thirty-nine faculty members published a total of 640 publications during the period, representing an average of 16.4 publications per scientist. Among the 640 publications, 521 (81.4 %) were cited 9,204 times, an average of 14.4 citations per publication. More publications co-authored by the SBS faculty with foreign researchers (mostly from the U.S.A.) were published in mainstream journals than publications by three other co-authorship types. Accordingly, publications by international co-authorships received more citations compared to citation levels of three other co-authorship types in terms of the average citations per publication. The study has found a concentration effect, whereby quite a small number of publications received approximately one-third of the citation performance generated by the SBS faculty at SNU. The results demonstrate that the citation performance of the SBS at SNU can be influenced considerably by the presence and productivity of ‘star’ scientists.     

Academic inbreeding and research productivity and impact in Australian law schools

This study compares the research productivity and impact of inbred and non-inbred faculty employed at Australian law schools. The sample consists of 429 academics, employed at 21 law schools. To measure research productivity and impact we use articles published in top law journals, defined in six different ways, as well as total citations and two different citation indices. We report results including, and excluding, publications in the academic’s home law review. We find evidence that silver-corded faculty outperform other faculty on one of the measures of publications in top journals, once the endogeneity of academic seniority, grant history and the status of the law school at which the individual is employed is addressed, but this finding is not robust across alternative measures of articles published in the top journals. We find that there is no statistically significant difference between the research productivity and impact of inbred and non-inbred faculty. This finding is robust to a range of different ways of measuring research productivity and impact and alternative econometric approaches, including using two-stage least squares to address the endogeneity of academic seniority, grant history and the status of the law school at which the legal academic is employed.     

Research production in the arts and humanities

This study explored the main factors influencing the research production in the arts and humanities. A questionnaire was constructed to identify and assess the effects of various factors important for the productivity of the individual researcher as reflected in the number of papers and Ph.D.'s produced. First, respondents were given the opportunity to list in their own words a number of important factors influencing research productivity. Secondly, they evaluated on rating scales the importance of a number of pre-selected factors (e.g. individual characteristics, organisational features, external factors) assumed to be important for research productivity. 50% of a sample of 256 researchers in the humanities responded. Ratings were grouped to produce a number of indices and these were subject to multiple regression analyses. The main results showed that the production of papers was predicted by the number of Ph.D.'s produced and inversely related to the importance of organisational factors. The production of Ph.D.'s was dependent on the year of the Ph.D. and the position of the respondent as well as on the number of papers s/he produced. A number of conclusions were drawn: a) there was support for the academic social position effect also in the humanities; b) organisational factors apparently played a minor role in comparison to individual characteristics in the humanities than in the sciences and; c) the differences in productivity of papers were also related to gender, but not to size, area or language of publications. Implications for further studies were suggested.This work was supported by a grant from the Council for Studies of Higher Education and finished while the first author was a Visiting Research Fellow at SPRU, University of Sussex, 1995. I wish to thankBen R. Martin, SPRU, University of Sussex andAnton Nederhof, CWTS, Leiden University for valuable comments on an earlier version of this paper which was presented at the workshop Studies on the Arts and Humanities and the Social Sciences, at SPRU, University of Sussex, 30 May, 1995.     

Analysis of bibliometric indicators for individual scholars in a large data set

Citation numbers and other quantities derived from bibliographic databases are becoming standard tools for the assessment of productivity and impact of research activities. Though widely used, still their statistical properties have not been well established so far. This is especially true in the case of bibliometric indicators aimed at the evaluation of individual scholars, because large-scale data sets are typically difficult to be retrieved. Here, we take advantage of a recently introduced large bibliographic data set, Google Scholar Citations, which collects the entire publication record of individual scholars. We analyze the scientific profile of more than 30,000 researchers, and study the relation between the h-index, the number of publications and the number of citations of individual scientists. While the number of publications of a scientist has a rather weak relation with his/her h-index, we find that the h-index of a scientist is strongly correlated with the number of citations that she/he has received so that the number of citations can be effectively be used as a proxy of the h-index. Allowing for the h-index to depend on both the number of citations and the number of publications, we find only a minor improvement.     

Italian sociologists: a community of disconnected groups

Examining coauthorship networks is key to study scientific collaboration patterns and structural characteristics of scientific communities. Here, we studied coauthorship networks of sociologists in Italy, using temporal and multi-level quantitative analysis. By looking at publications indexed in Scopus, we detected research communities among Italian sociologists. We found that Italian sociologists are fractured in many disconnected groups. The giant connected component could be split into five main groups with a mix of three main disciplinary topics: sociology of culture and communication (present in two groups), economic sociology (present in three groups) and general sociology (present in three groups). By applying an exponential random graph model, we found that collaboration ties are mainly driven by the research interests of these groups. Other factors, such as preferential attachment, gender and affiliation homophily are also important, but the effect of gender fades away once other factors are controlled for. Our research shows the advantages of multi-level and temporal network analysis in revealing the complexity of scientific collaboration patterns.

     

The value of an overseas research trip

As an essential part of the academic environment, international scientific mobility draws considerable attention from researchers. Previous studies have indicated a strong relationship between scientific mobility and scientific output. However, few researchers have addressed the causality between them. The research questions in this study focused on how the international scientific mobilization of the researchers affects their number of international collaborations, their ability to get published at higher impact factor journals, the number of citations that they get. Based on the SCOPUS database of English language scientific journal articles, this paper revealed the causal effects of international scientific mobility of the researchers on their scientific productivity, collaborations, and impact on science using the synthetic control method. The author’s affiliation on their articles provided the geographical location that can be tracked in time to infer the international scientific mobility of each author. A sample of more than 79,000 immobile scientists was used to create the synthetic versions of over 1500 internationally mobile scientists, so that, the synthetic version of each mobile author best resembled the academic ability of her/his counterpart mobile author in the pre-mobilization period. This allowed investigating the effects of the international mobilization on their publications by comparing the post-mobilization publication characteristics of the mobile authors and their immobile synthetic controls.The findings show strong evidence of a substantial positive effect of scientific mobility on the ability to get published in more prestigious journals, the number of citations received in total and from overseas, and international collaborations. The magnitude of the effect is conditional on the duration of scientific mobility.

     

Evolution of the publications in clinical neurology: scientific impact of different countries during the 2000–2009 period

We analyzed the productivity and visibility of publications on the subject category of Clinical Neurology by countries in the period 2000–2009. We used the Science Citation Index Expanded database of the ISI Web of Knowledge. The analysis was restricted to the citable documents. Bibliometric indicators included the number of publications, the number of citations, the median and interquartile range of the citations, and the h-index. We identified 170,483 publications (84.9 % original articles) with a relative increase of 28.5 % throughout the decade. Fourteen countries published over 2,000 documents in the decade and received more than 50,000 citations. The average of citations received per publication was 8 (interquartile range: 3–20) and the h-index was 261. USA was the country with the highest number of publications, followed by Germany, Japan, the UK and Italy. Moreover, USA publications had the largest number of citations received (44.5 % of total), followed by the UK, Germany, Canada, and Italy. On the other hand, Sweden, the Netherlands and the UK had the highest median citations for their total publications. During the period 2000–2009 there was a significant increase in Clinical Neurology publications. Most of the publications and citations comprised 14 countries, with the USA in the first position. Interestingly, most of the publications and citations originated from only 14 countries, with European countries with relatively low population, such as Switzerland, Austria, Sweden, Belgium, and the Netherlands, in this top group.     

Testing bibliometric indicators by their prediction of scientists promotions

We have developed a method to obtain robust quantitative bibliometric indicators for several thousand scientists. This allows us to study the dependence of bibliometric indicators (such as number of publications, number of citations, Hirsch index...) on the age, position, etc. of CNRS scientists. Our data suggests that the normalized h-index (h divided by the career length) is not constant for scientists with the same productivity but different ages. We also compare the predictions of several bibliometric indicators on the promotions of about 600 CNRS researchers. Contrary to previous publications, our study encompasses most disciplines, and shows that no single indicator is the best predictor for all disciplines. Overall, however, the Hirsch index h provides the least bad correlations, followed by the number of papers published. It is important to realize however that even h is able to recover only half of the actual promotions. The number of citations or the mean number of citations per paper are definitely not good predictors of promotion. Due to space constraints, this paper is a short version of a more detailed article. [JENSEN & AL., 2008B]     

Does self-citation pay?

Self-citations — those where authors cite their own works — account for a significant portion of all citations. These self-references may result from the cumulative nature of individual research, the need for personal gratification, or the value of self-citation as a rhetorical and tactical tool in the struggle for visibility and scientific authority. In this article we examine the incentives that underlie self-citation by studying how authors’ references to their own works affect the citations they receive from others. We report the results of a macro study of more than half a million citations to articles by Norwegian scientists that appeared in the Science Citation Index. We show that the more one cites oneself the more one is cited by other scholars. Controlling for numerous sources of variation in cumulative citations from others, our models suggest that each additional self-citation increases the number of citations from others by about one after one year, and by about three after five years. Moreover, there is no significant penalty for the most frequent self-citers — the effect of self-citation remains positive even for very high rates of self-citation. These results carry important policy implications for the use of citations to evaluate performance and distribute resources in science and they represent new information on the role and impact of self-citations in scientific communication.     

Research evaluation. Part I: productivity and citedness of a German medical research institution

An evaluation exercise was performed involving 313 papers of research staff (66 persons) of the Deutsche Rheuma-Forschungszentrum (DRFZ) published in 2004?C2008. The records and citations to them were retrieved from the Web of Science (Thomson Reuters) in March 2010. The authors compared productivity and citedness of ??group leaders?? vs. ??regular scientists??, of ??male scientists?? vs. ??female scientists?? using citation-based indexes. It was found that ??group leaders?? are more prolific and cited more often than ??regular scientists??, the same is true considering ??male?? vs. ??female scientists??. The greatest contrast is observed between ??female leaders?? and ??female regular scientists??. The above mentioned differences are significant in indexes related to the number of papers, while values of indexes characterizing the quality of papers (average citation rate per paper and similar indexes) are not substantially different among the groups compared. The mean value of percentile rank index for all the 313 papers is 58.5, which is significantly higher than the global mean value of about 50. This fact is evidence of a higher citation status, on average, of the publications from the DRFZ.     

A multidimensional approach to academic productivity

This study represents one of the first attempts to use empirical analysis to estimate academic productivity complex and proves the thesis that academic productivity is a function of multidimensional combination of the work of academic researchers: the scientific work, education, and external relationships. Given the complexity of academic productivity, it is necessary to clarify that it is divided into scientific productivity of the first type (scientific publications); scientific productivity of the second type (awards and academic positions); productivity in terms of external relationships (or external advice); and educational productivity. This objective of this paper is achieved through a sample survey (2,738 academics responded) conducted by Italian researchers from the PIR research project. The results obtained, however (as a case of estimates obtained using the results of a sample survey), are the result of a working reality that Italian academics are flooded by a myriad of activities that are not always consistent with the primary aims of the work of a researcher with an organisational and environmental well-being at the limit of iper productivity (or hyper productivity). The overall productivity (academic productivity) is significantly correlated with the four dimensions: average annual scientific productivity of the first type, average annual scientific productivity of the second type, the productivity external advice and, lastly, teaching productivity. The estimate of the sizes for the four indicators of productivity are the result of a literature search of the primary techniques used to assess productivity in academia. By comparing the most significant indicators, we managed to select all of the technical aspects missing in the Italian system of evaluation. This process allowed for us to add additional variables characterising the various aspects of productivity and prove the validity of our theory about the multidimensionality of academic productivity.