Do citations and impact factors relate to the real numbers in publications? A case study of citation rates, impact, and effect sizes in ecology and evolutionary biology

Metrics of success or impact in academia may do more harm than good. To explore the value of citations, the reported efficacy of treatments in ecology and evolution from close to 1,500 publications was examined. If citation behavior is rationale, i.e. studies that successfully applied a treatment and detected greater biological effects are cited more frequently, then we predict that larger effect sizes increases study relative citation rates. This prediction was not supported. Citations are likely thus a poor proxy for the quantitative merit of a given treatment in ecology and evolutionary biology—unlike evidence-based medicine wherein the success of a drug or treatment on human health is one of the critical attributes. Impact factor of the journal is a broader metric, as one would expect, but it also unrelated to the mean effect sizes for the respective populations of publications. The interpretation by the authors of the treatment effects within each study differed depending on whether the hypothesis was supported or rejected. Significantly larger effect sizes were associated with rejection of a hypothesis. This suggests that only the most rigorous studies reporting negative results are published or that authors set a higher burden of proof in rejecting a hypothesis. The former is likely true to a major extent since only 29 % of the studies rejected the hypotheses tested. These findings indicate that the use of citations to identify important papers in this specific discipline—at least in terms of designing a new experiment or contrasting treatments—is of limited value.     

Why does library holding format really matter for book impact assessment?: Modelling the relationship between citations and altmetrics with print and electronic holdings

Scientometrics - Scholarly books are important outputs in some fields and their many publishing formats seem to introduce opportunities to scrutinize their impact. As there is a growing interest in...     

Can nanotechnology be ‘green’? Comparing efficacy of nano and microparticles in cementitious materials

The use of nano-sized particles in cementitious materials introduces a myriad of potential innovations from new functionality to enhanced mechanical performance, but such materials can be energy-intensive to manufacture. With increasing emphasis on sustainable development, it is important to investigate and understand benefits and costs of using nanomaterials compared to relatively less energy-intensive microparticles. The current research investigates the effect of chemically inert nano and microparticles (i.e., titanium dioxide (TiO₂) and calcium carbonate (limestone)) on early age properties and behavior of cement-based materials. Results indicate that the early age hydration rates, shrinkage, and pore structure of cement-based materials can be modified and optimized by tailoring the size of fillers. Life cycle analysis indicates that photocatalytic reactivity of TiO₂ could offset initial higher environmental impacts. Thus, optimally sized nanoparticles could revolutionize the construction industry by allowing tailoring of structure and properties of cement-based composites, with environmental sustainability preserved through the selection of lower embodied-energy particles.     

What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018

Scientometrics - The purpose of this paper is to update the review of Bornmann and Daniel (J Doc 64(1):45–80, 2008) presenting a narrative review of studies on citations in scientific...     

Re-examine the relationship between patents and Tobin’s q

Since the relationship between patents and Tobin’s q is confusing, this paper utilizes panel threshold regression model to re-examine the relationship between patent counts/sales and Tobin’s q. This study finds out patent citations/sales has a single threshold effect on the relationship between patent counts/sales and Tobin’s q in the US pharmaceutical industry. The single threshold value of patent citations/sales is 328.81, and it divides the value of patent citations/sales into two regimes: the first regime (patent citations/sale ≦ 328.81) and the second regime (patent citations/sale > 328.81). The results indicate that patent counts/sales positively affect Tobin’s q in the two regimes. In addition, this study demonstrates that the extent of the positive effect of patent counts/sales on Tobin’s q is different. This study verifies that patent citations/sales moderates the relationship between patent counts/sales and Tobin’s q. Once patent citations/sales is below the threshold value, the extent of the positive relationship between patent counts/sales and Tobin’s q is the most. Therefore, this study finds out that the first regime is optimal.     

Publication modalities ‘article in press’ and ‘open access’ in relation to journal average citation

Scientometrics - There has been a generalization in the use of two publication practices by scientific journals during the past decade: (1) ‘article in press’ or early view, which...     

How do latecomers catch up with forerunners? Analysis of patents and patent citations in the field of flat panel display technologies

This paper sets out to explore the patterns of technological change and knowledge spillover in the field of flat panel display (FPD) technology, along with the catching-up behavior of latecomers, through the analysis of US patents and patent citations between 1976 and 2005. Our results show that: (i) the catching-up by FPD technology latecomers began at the transition stage (1987-1996) when the dominant design became established in areas with high ‘revealed technology advantage’ (RTA); (ii) there is no apparent localization of knowledge spillover amongst FPD technology latecomers; instead, higher citation frequencies of forerunners’ patents were found in latecomers’ FPD patents during the transition (1987–1996) and post-dominant design (1997–2005) stages and; and (iii) a few extraordinary peaks were found in the citation frequency of forerunners’ patents at long citation lags in latecomers’ FPD patents, particularly during the transition stage (1987–1996), indicative of the knowledge threshold which latecomers need to cross in order to catch up with forerunners.     

The impact assessment ‘arms race’ and visions for the future

In this commentary on Morrison-Saunders et al.'s [Morrison-Saunders A, Pope J, Gunn JAE, Bond A, Retief F. 2014. Strengthening impact assessment: a call for integration and focus. Impact Assess Project Appraisal. 44] position paper we focus on two main issues: their (1) call for unification and (2) assessment of the implications for impact assessment (IA) of the better regulation agenda. The importance of names and naming, plus the existence of genuine differences between IA tools and IA communities lead us to problematize the goal of unification. We argue that their ‘call to arms’ is based on a partial analysis of drivers of deregulation and is unnecessarily alarmist. Our commentary concludes with recommendations for a future agenda that prioritizes creativity arising from a vibrant community of practice.     

Multiple regression analysis of a patent’s citation frequency and quantitative characteristics: the case of Japanese patents

Although many studies have been conducted to clarify the factors that affect the citation frequency of “academic papers,” there are few studies where the citation frequency of “patents” has been predicted on the basis of statistical analysis, such as regression analysis. Assuming that a patent based on a variety of technological bases tends to be an important patent that is cited more often, this study examines the influence of the number of cited patents’ classifications and compares it with other factors, such as the numbers of inventors, classifications, pages, and claims. Multiple linear, logistic, and zero-inflated negative binomial regression analyses using these factors are performed. Significant positive correlations between the number of classifications of cited patents and the citation frequency are observed for all the models. Moreover, the multiple regression analyses demonstrate that the number of classifications of cited patents contributes more to the regression than do other factors. This implies that, if confounding between factors is taken into account, it is the diversity of classifications assigned to backward citations that more largely influences the number of forward citations.     

Should we use the mean citations per paper to summarise a journal’s impact or to rank journals in the same field?

The mean citations per paper is used increasingly as a simple metric for indicating the impact of a journal or comparing journal rankings. While convenient, we suggest that it has limitations given the highly skewed distributions of citations per paper in a wide range of journals.     

Can tissue weighting factors be established for the embryo and fetus?

For the calculation of effective dose (E), tissue weighting factors (wT) are needed to represent the varying radiosensitivity of the tissues in the human body with respect to the induction of stochastic effects. The wT-values have been determined by the International Commission on Radiological Protection according to the stochastic detriment of human populations during their postnatal life. This study discusses whether these wT-values can also be used for the embryo/fetus. For this purpose, the epidemiological data and some results from animal experiments on carcinogenesis after prenatal radiation exposures have been reviewed. Most human data have been obtained from studies of childhood cancers (<19 y of age) after exposures during prenatal development. These tumours differ from those observed later in life after radiation exposures of children and adults. From animal data and more recent results from the atomic bomb survivors, it appears that not only childhood cancers but also cancers occurring during adulthood would have to be considered for the determination of possible wT-values after prenatal irradiation. From the present data it is concluded that sufficient data for defining wT-values following exposure of the embryo/fetus are not available at present.     

Does gender moderate the relationship between driver aggression and its risk factors?

Aim

The current study assessed gender as a potential moderator of the relationship between self-reported driver aggression and various demographic variables, general and driving-related risk factors.

Methods

Using data from a general-population telephone survey conducted from July 2002 through June 2005, two approaches to binary logistic regression were adopted. Based on the full dataset (n = 6259), the initial analysis was a hierarchical-entry regression examining self-reported driver aggression in the last 12 months. All demographic variables (i.e., gender, age, income, education, marital status), general risk factors (i.e., psychological distress, binge drinking, cannabis use), and driving-related risk factors (i.e., driving exposure, stressful driving, exposure to busy roads, driving after drinking, driving after cannabis use) were entered in the first block, and all two-way interactions with gender were entered stepwise in the second block. The subsequent analysis involved dividing the sample by gender and conducting logistic regressions with main effects only for males (n = 2921) and females (n = 3338) separately.

Results

Although the prevalence of driver aggression in the current sample was slightly higher among males (38.5%) than females (32.9%), the difference was small, and gender did not enter as a significant predictor of driver aggression in the overall logistic regression. In that analysis, difficulty with social functioning and being older were associated with a reduced risk of driver aggression. Marital status and education were unrelated to aggression, and all other variables were associated with an increased risk of aggression. Gender was found to moderate the relationships between driver aggression and only three variables: income, psychological distress, and driving exposure. Separate analyses on the male and female sub-samples also found differences in the predictive value of income and driving exposure; however, the difference for psychological distress could not be detected using this separate regression approach. The secondary analysis also identified slight differences in the predictive value of four of the risk factors, where the odds ratios for both males and females were in the same direction but only one of the two was statistically significant.

Conclusions

The results demonstrate the importance of conducting the gender analysis using both regression approaches. With few exceptions, factors that were predictive of driver aggression were generally the same for both male and female drivers.     

Can the impact of grey literature be assessed? An investigation of UK government publications cited by articles and books

Scientometrics - Grey literature encompasses a range of relatively informal textual outputs that are not indexed in citation databases. Although they are usually ignored in research evaluations, it...     

What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines

Scientometrics - The goal of the study here is to model and analyze the relation between research funding and citation-based performance in science to predict the diffusion of new scientific...     

Is there a gender-based fragmentation of communication science? An investigation of the reasons for the apparent gender homophily in citations

Multiple studies report that male scholars cite publications of male authors more often than their female colleagues do—and vice versa. This gender homophily in citations points to a fragmentation of science along gender boundaries. However, it is not yet clear whether it is actually (perceived) gender characteristics or structural conditions related to gender that are causing the heightened citation frequency of same-sex authors. A bibliometric study on the two leading German communication science journals Publizistik and Medien & Kommunikationswissenschaft was employed to further analyze the causes of the phenomenon. As scholars tend to primarily cite sources from their own area of research, differences among male and female scholars regarding their engagement in certain research fields become relevant. It was thus hypothesized that the research subject might mediate the relationship between the citing and cited authors’ genders. A first analysis based on n = 917 papers published in the period from 1970 to 2009 confirmed the expected gender-differences regarding research-activity in certain fields. Subsequently, structural equation modeling was employed to test the suggested mediation model. Results show the expected mediation to be a complementary one indicating that gender homophily in citations is partly due to topical boundaries. While there are alternative explanations for the remaining direct effect, it may suggest that a fragmentation of science along gender boundaries is indeed an issue that communication science must face.     

Where and what do they publish? Editors’ and editorial board members’ affiliated institutions and the citation counts of their endogenous publications in the field of communication

Scientometrics - This study examined the geographical diversity and publication patterns of editors and editorial board members in communication journals. The results indicated that the diversity...