A multidimensional approach to academic productivity

This study represents one of the first attempts to use empirical analysis to estimate academic productivity complex and proves the thesis that academic productivity is a function of multidimensional combination of the work of academic researchers: the scientific work, education, and external relationships. Given the complexity of academic productivity, it is necessary to clarify that it is divided into scientific productivity of the first type (scientific publications); scientific productivity of the second type (awards and academic positions); productivity in terms of external relationships (or external advice); and educational productivity. This objective of this paper is achieved through a sample survey (2,738 academics responded) conducted by Italian researchers from the PIR research project. The results obtained, however (as a case of estimates obtained using the results of a sample survey), are the result of a working reality that Italian academics are flooded by a myriad of activities that are not always consistent with the primary aims of the work of a researcher with an organisational and environmental well-being at the limit of iper productivity (or hyper productivity). The overall productivity (academic productivity) is significantly correlated with the four dimensions: average annual scientific productivity of the first type, average annual scientific productivity of the second type, the productivity external advice and, lastly, teaching productivity. The estimate of the sizes for the four indicators of productivity are the result of a literature search of the primary techniques used to assess productivity in academia. By comparing the most significant indicators, we managed to select all of the technical aspects missing in the Italian system of evaluation. This process allowed for us to add additional variables characterising the various aspects of productivity and prove the validity of our theory about the multidimensionality of academic productivity.     

Research or management? An investigation of the impact of leadership roles on the research performance of academic administrators

This study quantitatively investigated whether the research performance of academic administrators (leaders) was affected by their administrative services. We sampled 111 academic administrators, including 90 department deans and 21 university presidents, from 26 universities. These leaders’ research performance was determined by the number of their publications and citations in the Web of Science databases. To compare the effect on their research performance by their administrative roles, we proposed four periods: the pre-position period, the latest position period, the reference period, and the in-position period. Statistical methods were applied to compare the research performance of the sampled administrators before and after they accepted the current administrative roles. The results suggest that 80% of academic leaders’ productivity and citation have fallen by 42 and 62% averagely. The extent of such impact varied in different disciplines. Leaders’ performance in medicine fields has declined the most (60%). The impact on research productivity appeared to be stronger for administrators serving in higher-ranking universities. In addition, the research performance of both university presidents and department deans were substantially influenced by their administrative services.     

Research productivity of scientists in consolidated vs. non-consolidated teams: The case of Spanish university geologists

We present some results of an evaluation of research performance of Spanish senior university researchers in Geology. We analyse to what extent productivity of individual researchers is influenced by the level of consolidation of the team they belong to. Methodology is based on the combination of a mail survey carried out among a defined set of researchers, and a bibliometric study of their scientific output. Differences among researchers have been investigated with regard to team size and composition, patterns of publication in domestic and foreign journals, productivity, co-authorship of papers, and impact of publications. Results indicate that not belonging to a research team represents a handicap at the time of publishing in top international journals. Researchers belonging to consolidated teams are more productive than their colleagues in non-consolidated teams, and these in turn more than individuals without team. Team size does not appear to be as important for scientific productivity as the number of researchers within the team that reached a stable job position. Analysis of the impact factor of journals has not revealed differences among researchers with regard to the visibility of their papers. This revised version was published online in June 2006 with corrections to the Cover Date.     

Evaluation of Moroccan research using a bibliometric-based approach: investigation of the validity of the h-index

This paper presents the results of an evaluation of the national research system in Morocco. The exercise focuses on the period 1997–2006 and includes a comparison with South Africa, Egypt, Nigeria, Tunisia, Algeria, Portugal and Greece. Ratings of highly ranked researchers are developed on the basis of their number of publications, number of citations and also their ‘h-index’ (or Hirsch index). Finally, we examine the empirical model set by Glänzel that related the h-index to the number of publications and the mean citation rate per paper for these ‘upper-class’ researchers. The use of this model confirms that the h-index is likely to reflect the importance and the quality of the scientific output of a given researcher.     

Gender and productivity differentials in science

The paper presents the results of an examination of gender differences in scientific productivity on a sample of 840 respondents, half the young scientific population in Croatia. In the last decade gender differences in the scientific productivity of young researchers have increased, which may be the result of introducing a more competitive scientific system. Young female researchers publish an average of two scientific papers less than their male counterparts in five years, and their publications reach 70.6% of males" publication productivity in the same period. In the case of both sexes, about 15% researchers publish about half of all research papers, but even the most productive women publish less than their male counterparts Socio-demographic, educational and qualificational predictors contribute more or less equally to the number of scientific publications by women and men. It is not until we introduce structural variables that a strong sex differentiation appears because these factors are much more powerful in explaining the production of women. They show that female scientists" publication productivity is more strongly influenced by their position in the social organization of science. There are also considerable sex differences in the case of individual productivity predictors. International contacts determine the number of papers by female scientists most of all. Attendance at scientific conferences abroad is the most powerful predictor of male productivity, too, but reviewing colleagues" papers and academic degree are also very important. This revised version was published online in June 2006 with corrections to the Cover Date.     

“The Triple-Helix collaboration: Why do researchers collaborate with industry and the government? What are the factors that influence the perceived barriers?”

This paper addresses four questions: What is the extent of the collaboration between the natural sciences and engineering researchers in Canadian universities and government agencies and industry? What are the determinants of this collaboration? Which factors explain the barriers to collaboration between the university, industry and government? Are there similarities and differences between the factors that explain collaboration and the barriers to collaboration? Based on a survey of 1554 researchers funded by the Natural Sciences and Engineering Research Council of Canada (NSERC), the results of the multivariate regressions indicate that various factors explain the decision of whether or not to collaborate with industry and the government. The results also differed according to the studied fields. Overall, the results show that the variables that relate to the researcher’s strategic positioning, to the set-up of strategic networks, to the costs related to the production of the transferred knowledge and transactions explain in large part the researcher’s collaboration. The results of the linear regression pointed to various factors that affect collaboration with researchers: research budget, university localization, radicalness of research, degree of risk-taking culture and researcher’s publications. Finally, the last part of the paper presents the results, and what they imply for future research and theory building.     

How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations

Although bibliometrics has been a separate research field for many years, there is still no uniformity in the way bibliometric analyses are applied to individual researchers. Therefore, this study aims to set up proposals how to evaluate individual researchers working in the natural and life sciences. 2005 saw the introduction of the h index, which gives information about a researcher’s productivity and the impact of his or her publications in a single number (h is the number of publications with at least h citations); however, it is not possible to cover the multidimensional complexity of research performance and to undertake inter-personal comparisons with this number. This study therefore includes recommendations for a set of indicators to be used for evaluating researchers. Our proposals relate to the selection of data on which an evaluation is based, the analysis of the data and the presentation of the results.     

Characteristics and determinants of eminent scientists' productivity

The empirical research on the sample of 385 eminent Croatian scientists was carried out in order to explore the patterns and factors of their scientific productivity. The study design made it possible to compare the results with those obtained in the 1990 survey on a sample of the research population. The average scientific productivity of eminent researchers is not only several times larger but also shows a more intensive scientific collaboration and orientation towards the international scientific arena. The most important predictors of the elite's productivity are also qualificational and organizational variables but of a more selective nature. By including the eminent scientists' gatekeeping roles, the explanation of their total, co-authored and foreign publications can be improved.     

A bibliometric analysis of the effectiveness of Korea’s Biotechnology Stimulation Plans,with a comparison with four other Asian nations

This study investigates the scientific output and publication patterns of Korean biotechnology before and after the start of the Korean Biotechnology Stimulation Plans (1994–2007), and then compares the results with publication data from the same time periods for Japan, the People’s Republic of China, Taiwan and Singapore. For this study, 14,704 publications, published by at least one researcher from one of the five Asian nations (indexed by SCI Expanded during the years 1990–1993 and the years 2000–2003), were considered. A marked increase of Korean research output in biotechnology was largely influenced by an increasing tendency for researchers to enter the field of biotechnology and by increased expenditures for R&D activity through the Korean Biotechnology Stimulation Plans. In addition, the SCI Expanded coverage of national journals affected the scientific output and publication patterns of Japanese and Korean researchers. Looking at the Korean publications by collaboration type, international collaboration leads to more publications in mainstream journals of high impact factors than local and domestic collaborations for the two periods. However, although the Korean Biotechnology Stimulation Plans were followed by a remarkable increase in South Korea’s research output, this increase has not been accompanied by growth in the quality of those publications in terms of impact factors of journals for Korean publications.     

Gender gaps in international research collaboration: a bibliometric approach

This paper addresses gender differences in international research collaboration measured through international co-authorship. The study is based on a dataset consisting of 5600 Norwegian researchers and their publication output during a 3-year period (44,000 publications). Two different indicators are calculated. First, the share of researchers that have been involved in international collaboration as measured by co-authorship, and second, the share of their publications with international co-authorship. The study shows that the field of research is by far the most important factor influencing the propensity to collaborate internationally. There are large differences from humanities on the one hand, where international collaboration in terms of co-authorship is less common, to the natural sciences on the other, where such collaboration is very frequent. On an overall level, we find distinct gender differences in international research collaboration in Norway in the favour of men. However, men and women are not equally distributed across fields and there are relatively more female researchers in fields where the international collaboration rates generally are lower. When the data are analysed by scientific field, academic position, and publication productivity of the researchers, the gender differences in the propensity to collaborate with colleagues in other countries are minor only, and not statistically significant. Concerning gender inequality in science, the main challenge seems to be the lower productivity level of female researchers, which obviously hinders their academic career development. Differences in international collaboration are unlikely to be an important factor in this respect, at least not in the Norwegian research context analysed in this study.

     

Evaluating top faculty researchers and the incentives that motivate them

We survey tenure-track faculty members employed in three fields in colleges of agriculture at land-grant universities—agricultural economics, agronomy, and food science—to evaluate the effects of different employment structures and incentives on research productivity. These evaluations include conducting statistical tests to assess any effects of different academic appointments and developing a regression model to measure the effects of these and other attributes on individual research productivity, as defined by the number of publications in the Thomson ISI Web of Science. We find faculty who hold larger teaching and extension appointments produce fewer publications; we also find positive effects on the number of publications for grants and university funding, multi-institutional research collaboration, and number of graduate students advised.     

The value of an overseas research trip

As an essential part of the academic environment, international scientific mobility draws considerable attention from researchers. Previous studies have indicated a strong relationship between scientific mobility and scientific output. However, few researchers have addressed the causality between them. The research questions in this study focused on how the international scientific mobilization of the researchers affects their number of international collaborations, their ability to get published at higher impact factor journals, the number of citations that they get. Based on the SCOPUS database of English language scientific journal articles, this paper revealed the causal effects of international scientific mobility of the researchers on their scientific productivity, collaborations, and impact on science using the synthetic control method. The author’s affiliation on their articles provided the geographical location that can be tracked in time to infer the international scientific mobility of each author. A sample of more than 79,000 immobile scientists was used to create the synthetic versions of over 1500 internationally mobile scientists, so that, the synthetic version of each mobile author best resembled the academic ability of her/his counterpart mobile author in the pre-mobilization period. This allowed investigating the effects of the international mobilization on their publications by comparing the post-mobilization publication characteristics of the mobile authors and their immobile synthetic controls.The findings show strong evidence of a substantial positive effect of scientific mobility on the ability to get published in more prestigious journals, the number of citations received in total and from overseas, and international collaborations. The magnitude of the effect is conditional on the duration of scientific mobility.

     

Variation in research collaboration patterns across academic ranks

The ability to activate and manage effective collaborations is becoming an increasingly important criteria in policies on academic career advancement. The rise of such policies leads to development of indicators that permit measurement of the propensity to collaborate for academics of different ranks, and to examine the role of several variables in collaboration, first among these being the researchers’ disciplines. In this work we apply an innovative bibliometric approach based on individual propensity for collaboration to measure the differences in propensity across academic ranks, by discipline and for choice of collaboration forms—intramural, extramural domestic and international. The analysis is based on the scientific production of Italian academics for the period 2006–2010, totaling over 200,000 publications indexed in Web of Science. It shows that assistant professors register a propensity for intramural collaboration that is clearly greater than for professors of higher ranks. Vice versa, the higher ranks, but not quite so clearly, register greater propensity to collaborate at the international level.     

To include or not: the role of review papers in citation-based analysis

Citation information helps researchers observe the evolution of the knowledge. In scientific publications, a review paper discusses a professional field and thus tends to have more citations than general papers do. This study investigated whether specific characteristics of review papers induce different results in citation-based analysis. From the Scopus database, we collected scientific publications in a specific research field, e-tourism, to examine the role of review papers in citation-based analysis. The dataset includes 1421 publications covering the period from the 1988–2015. Empirical statistics show that review papers’ specific citation patterns influence citation analysis. First, in the main path analysis, the result expresses review papers’ integrative role in linking papers from diverse perspectives toward a clear mainstream topic. Second, in a well-defined research context, review papers introduce bias in citation-based clustering analysis because the specific high citation pattern in review papers obfuscates the grouping process. When using citation information in analysis, scholars must consider the purpose of the study and treat review papers distinctly to avoid bias when using certain analysis methods and datasets.     

Dispersion measures for h-index: a study of the Brazilian researchers in the field of mathematics

Although the h-index is considered a significant indicator to evaluate the researchers' performance, as it simultaneously measures aspects related to their scientific productivity and citation impact, several studies have pointed out its deficiencies and limitations. In this context, this study aims to evaluate the contribution of dci and dco indicators, two Hirsch-type indices, to measure the reasonableness of the h-index as a representation of the researcher’s scientific performance. The universe of analysis consisted of 116 Brazilian mathematicians holding CNPq grants. For each researcher, the number of articles, the number of citations per article and the year of publication were collected in the Scopus database. Then, for each researcher, the h-index and dci and dco indices were calculated. The dci and dco indicators allowed selective and productive researchers to be to distinguished more accurately. In addition, they contributed to estimate the possibility of the researcher to increase his/her h-index. The study concludes that the dci and dco indices were able to reliably measure the distribution dispersion of the researchers' citation impact together with their h-index. Consequently, they managed to estimate the representativeness of h-index as an indicator of the researchers' scientific performance in the field of mathematics in Brazil.

     

How does scientific success relate to individual and organizational characteristics? A scientometric study of psychology researchers in the German-speaking countries

Purpose: To provide up-to-date bibliometric reference data describing the output and success of psychology researchers in the German-speaking countries, including lifetime publication and citation numbers, and to investigate associations of bibliometric measures with academic status and gender as well as the department characteristics of size and quota of senior researchers. Method Queried literature databases using an extensive online register of academic psychologists in the German-speaking countries, obtaining valid data for 85 % (N = 1742) of the population of interest. Findings Skewed distributions for publications and citations; maximum number of German-language (=native) publications much higher than maximum number of English-language publications; relatively large part of population publishing almost exclusively in German; publication count predictable by academic status, gender, department size, and quota of senior researchers; citation count predictable by publication count, status, department size, and quota of senior researchers; department characteristics interact with individual characteristics to produce specific conditions under which publication count and citation count are higher or lower than expected: combination of female gender, small department size and large quota of senior researchers is associated with particularly increased publication count; female gender and large department size are associated with decreased publication count; high publication count, large department size and low quota of senior researchers are associated with increased citation count; low publication count and large quota of senior researchers are associated with decreased citation count. Conclusions Reference values for scientific output provided in this study provide an anchor for monitoring and international comparison; despite considerable noise in data, we show that interactions of individual and organizational characteristics are relevant for scientific success and should be investigated further, e.g. by adopting various measures of organizational diversity and tracing a population longitudinally.     

A ‘perverse incentive’ from bibliometrics: could National Research Assessment Exercises (NRAEs) restrict literature availability for nature conservation?

National Research Assessment Exercises (NRAEs) aim to improve returns from public funding of research. Critics argue that they undervalue publications influencing practice, not citations, implying that journals valued least by NRAEs are disproportionately useful to practitioners. Conservation biology can evaluate this criticism because it uses species recovery plans, which are practitioner-authored blueprints for recovering threatened species. The literature cited in them indicates what is important to practitioners’ work. We profiled journals cited in 50 randomly selected recovery plans from each of the USA, Australia and New Zealand, using ranking criteria from the Australian Research Council and the SCImago Institute. Citations showed no consistent pattern. Sometimes higher ranked publications were represented more frequently, sometimes lower ranked publications. Recovery plans in all countries also contained 37 % or more citations to ‘grey literature’, discounted in NRAEs. If NRAEs discourage peer-reviewed publication at any level they could exacerbate the trend not to publish information useful for applied conservation, possibly harming conservation efforts. While indicating the potential for an impact does not establish that it occurs, it does suggest preventive steps. NRAEs considering the proportion of papers in top journals may discourage publication in lower-ranked journals, because one way to increase the proportion of outputs in top journals is by not publishing in lower ones. Instead, perhaps only a user-nominated subset of publications could be evaluated, a department’s or an individual’s share of the top publications in a field could be noted, or innovative new multivariate assessments of research productivity applied, including social impact.     

$$C^3$$-index: a PageRank based multi-faceted metric for authors’ performance measurement

Ranking scientific authors is an important but challenging task, mostly due to the dynamic nature of the evolving scientific publications. The basic indicators of an author’s productivity and impact are still the number of publications and the citation count (leading to the popular metrics such as h-index, g-index etc.). H-index and its popular variants are mostly effective in ranking highly-cited authors, thus fail to resolve ties while ranking medium-cited and low-cited authors who are majority in number. Therefore, these metrics are inefficient to predict the ability of promising young researchers at the beginning of their career. In this paper, we propose \(C^3\)-index that combines the effect of citations and collaborations of an author in a systematic way using a weighted multi-layered network to rank authors. We conduct our experiments on a massive publication dataset of Computer Science and show that—(1) \(C^3\)-index is consistent over time, which is one of the fundamental characteristics of a ranking metric, (2) \(C^3\)-index is as efficient as h-index and its variants to rank highly-cited authors, (3) \(C^3\)-index can act as a conflict resolution metric to break ties in the ranking of medium-cited and low-cited authors, (4) \(C^3\)-index can also be used to predict future achievers at the early stage of their career.     

The law of limited excellence: publication productivity of Israel Prize laureates in the life and exact sciences

The present paper extends Lotka’s theorem—which we rename as “the law of limited excellence”—while empirically modelling the scientific productivity of 46 Israel Prize laureates in the life and exact sciences—a group best described as ‘Star Scientists’. By focusing on this highly selective group we expose unequal scientific productivity even amongst Israel’s most prolific scientists. Specifically, we test the invariance of Lotka’s law by focusing attention on the extreme tail of publication distributions while empirically exploring the non-linearity of its seemingly “flat” tail. By exposing the rarity of excellence even in this extreme end of publication productivity we extend the generality of Lotka’s theorem and expose that—like a fractal—the tail of excellence behaves as the entire distribution. We end this empirical contribution by suggesting a few implications for research and policy.     

A new indicator for international visibility: exploring Brazilian scientific community

Brazilian science has increased fast during the last decades. An example is the increasing in the country’s share in the world’s scientific publication within the main international databases. But what is the actual weight of international publications to the whole Brazilian productivity? In order to respond this question, we have elaborated a new indicator, the International Publication Ratio (IPR). The data source was Lattes Database, a database organized by one of the main Brazilian S&;T funding agency, which encompasses publication data from 1997 to 2004 of about 51,000 Brazilian researchers. Influences of distinct parameters, such as sectors, fields, career age and gender, are analyzed. We hope the data presented may help S&;T managers and other S&;T interests to better understand the complexity under the concept scientific productivity, especially in peripheral countries in science, such as Brazil.