Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants

Summary We investigated committee peer review for awarding long-term fellowships to post-doctoral researchers as practiced by the Boehringer Ingelheim Fonds (B.I.F.) - a foundation for the promotion of basic research in biomedicine. Assessing the validity of selection decisions requires a generally accepted criterion for research impact. A widely used approach is to use citation counts as a proxy for the impact of scientific research. Therefore, a citation analysis for articles published previous to the applicants' approval or rejection for a B.I.F. fellowship was conducted. Based on our model estimation (negative binomial regression model), journal articles that had been published by applicants approved for a fellowship award (n = 64) prior to applying for the B.I.F. fellowship award can be expected to have 37% (straight counts of citations) and 49% (complete counts of citations) more citations than articles that had been published by rejected applicants (n = 333). Furthermore, comparison with international scientific reference values revealed (a) that articles published by successful and non-successful applicants are cited considerably more often than the “average” publication and (b) that excellent research performance can be expected more of successful than non-successful applicants. The findings confirm that the foundation is not only achieving its goal of selecting the best junior scientists for fellowship awards, but also successfully attracting highly talented young scientists to apply for B.I.F. fellowships.     

Field classification of publications in Dimensions: a first case study testing its reliability and validity

Dimensions is a research data infrastructure and tool, including grants, publications, citations, clinical trials, and patents in one place. An interesting feature of Dimensions is its field classification scheme, which is not based on journal classification systems, as in the Web of Science or Scopus, but on machine learning. Each publication is assigned to at least one field. Using the set of my own publications, I investigated whether they were reliably and validly assigned to fields. The results put in question the reliability and validity of the scheme. Large scale studies seem necessary to investigate the scheme in more detail.

     

Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation

Scientometrics - While bibliometrics are widely used for research evaluation purposes, a common theoretical framework for conceptually understanding, empirically studying, and effectively teaching...     

Discontinuities in citation relations among journals: self-organized criticality as a model of scientific revolutions and change

Scientometrics - Using 3-year moving averages of the complete Journal Citation Reports 1994–2016 of the Science Citation Index and the Social Sciences Citation Index (combined), we analyze...     

Electron spectrometer in adjustable triode configuration for photo-induced field emission measurements

We have constructed a new ultrahigh vacuum apparatus with a triode configuration for the systematic investigation of photo-induced field emission (PFE) from metallic or semiconducting cathodes. These are exposed to electric fields up to 400 MV∕m and laser irradiation by means of hole or mesh gates. Cathodes and gates are in situ exchangeable and adjustable with high precision to ensure a homogeneous extraction of electrons which are partially transmitted to the fixed electron spectrometer. Its hemispherical sector analyzer provides an energy resolution limit of 8 meV. The commissioning of the measurement system has been performed with a tungsten needle. Its temperature showed up in the high-energy tail of the electron spectrum, while its work function was derived from the spectral low-energy part combined with the integral current-voltage curve. First PFE measurements on B-doped Si-tip arrays yielded a small field emission current increase under green laser illumination. A shift and splitting of the energy spectra was observed which revealed different emission regimes as well as the photosensitivity of the cathode due to carrier excitation into the conduction band. For the full exploitation of the PFE system, a tunable laser over a wide eV-range is required.     

How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data

In this short communication, we provide an overview of a relatively newly provided source of altmetrics data which could possibly be used for societal impact measurements in scientometrics. Recently, Altmetric—a start-up providing publication level metrics—started to make data for publications available which have been mentioned in policy-related documents. Using data from Altmetric, we study how many papers indexed in the Web of Science (WoS) are mentioned in policy-related documents. We find that less than 0.5% of the papers published in different subject categories are mentioned at least once in policy-related documents. Based on our results, we recommend that the analysis of (WoS) publications with at least one policy-related mention is repeated regularly (annually) in order to check the usefulness of the data. Mentions in policy-related documents should not be used for impact measurement until new policy-related sites are tracked.     

A decade of in-text citation analysis based on natural language processing and machine learning techniques: an overview of empirical studies

Scientometrics - In-text citation analysis is one of the most frequently used methods in research evaluation. We are seeing significant growth in citation analysis through bibliometric metadata,...