Scientists' performance and consolidation of research teams in Biology and Biomedicine at the Spanish Council for Scientific Research

Summary Empirical evidence is given on how membership in a consolidated, well-established research team provides researchers with some competitive advantage as compared to their colleagues in non-consolidated teams. Data were obtained from a survey of researchers ascribed to the 'Biology and Biomedicine' area of the Spanish Council for Scientific Research, as well as from their curricula vitae. One quarter of the scientists work as members of teams in the process of consolidation. Our findings illustrate the importance, for the development and consolidation of research teams, of the availability of a minimum number of researchers with a permanent position and of a minimum number of support staff and non-staff personnel (mainly post-doctoral fellows). Consolidation of research teams has a clear influence on the more academic-oriented quantitative indicators of the scientific activity of individuals. Researchers belonging to consolidated teams perform quantitatively better than their colleagues in terms of the number of articles published in journals covered in the Journal Citation Reports, but not in terms of the impact of these publications. Consolidation favours publication, but not patenting, and it also has a positive effect on the academic prestige of scientists and on their capacity to train new researchers. It does not significantly foster participation in funded R&D projects, nor does it influence the establishment of international collaborations. Impact is influenced to a remarkable degree by seniority and professional background, and is significantly greater for young scientists who have spent time abroad at prestigious research laboratories.     

The effect of team consolidation on research collaboration and performance of scientists. Case study of Spanish university researchers in Geology

We analyse to what extent research collaboration and performance of individual scientists is influenced by the level of consolidation of the team they belong to. A case study of Spanish senior university researchers in Geology is performed. Methodology is based on the combination of a mail survey carried out among a defined set of researchers, and a bibliometric study of their scientific output. Results provide support for the hypothesis that consolidation of research teams would result in a greater facility to establish contacts and collaborations with colleagues, that could benefit all members of the team, fostering their participation in funded projects and favouring their potential to publish in international mainstream journals. This revised version was published online in June 2006 with corrections to the Cover Date.     

Computer science research: more production,less productivity

It is commonly accepted that scientific research or, more precisely, the number of scientific publications, in computer science has greatly increased over the last few years. The reason would appear to be the pressure to publish, coined by the expression ”Publish or perish”, which is, among other things, necessary for promotions and applications for grants or projects. In this paper we have conducted a study that covers computer science publications from 1936 to 2010 in order to quantify this increase in publications regarding computing research. We have considered the computing conferences and journals available in the DBLP computer science bibliography (DBLP 2013) database, including more than 1.5 million papers, and more than 4 million authors (more than 900,000 different people), corresponding to about 1,000 different journals and 3,000 different conferences and workshops. Our study confirms and quantifies these increases with regard to the number of papers, number of authors, number of papers per author, etc. However, it also reaches a surprising conclusion: the real productivity of researchers has decreased throughout history. The reason for this decrease is the average number of authors per paper, which has grown significantly and is currently three.     

Team consolidation,social integration and scientists’ research performance: An empirical study in the Biology and Biomedicine field

The effects of team consolidation and social integration on individual scientists’ activity and performance were investigated by analysing the relationships between these factors and scientists’ productivity, impact, collaboration patterns, participation in funded research projects and programs, contribution to the training of junior researchers, and prestige. Data were obtained from a survey of researchers ascribed to the Biology and Biomedicine area of the Spanish Council for Scientific Research, and from their curricula vitae. The results show that high levels of team consolidation and of integration of the scientist within his or her team are factors which might help create the most favourable social climate for research performance and productivity. Researchers who carried out their activity in a social climate characterized by these factors participated in more domestic research projects and supervised more doctoral dissertations than the rest of their colleagues. They were also more productive, as shown by the higher number of papers published in journals included in the Journal Citation Reports and the higher number of patents granted. These metrics are the main indicators taken into account in the evaluation of the research activity of Spanish scientists, and are therefore the activities that scientists invest the most energy in with a view to obtaining professional recognition. The results corroborate the importance of research teamwork, and draw attention to the importance of teamwork understood not as two or more scientists working together to solve a problem, but as a complex process involving interactions and interpersonal relations within a particular contextual framework     

Gender and productivity differentials in science

The paper presents the results of an examination of gender differences in scientific productivity on a sample of 840 respondents, half the young scientific population in Croatia. In the last decade gender differences in the scientific productivity of young researchers have increased, which may be the result of introducing a more competitive scientific system. Young female researchers publish an average of two scientific papers less than their male counterparts in five years, and their publications reach 70.6% of males" publication productivity in the same period. In the case of both sexes, about 15% researchers publish about half of all research papers, but even the most productive women publish less than their male counterparts Socio-demographic, educational and qualificational predictors contribute more or less equally to the number of scientific publications by women and men. It is not until we introduce structural variables that a strong sex differentiation appears because these factors are much more powerful in explaining the production of women. They show that female scientists" publication productivity is more strongly influenced by their position in the social organization of science. There are also considerable sex differences in the case of individual productivity predictors. International contacts determine the number of papers by female scientists most of all. Attendance at scientific conferences abroad is the most powerful predictor of male productivity, too, but reviewing colleagues" papers and academic degree are also very important. This revised version was published online in June 2006 with corrections to the Cover Date.     

Scientometrics of computer science research in India and China

Summary An analysis of 2058 papers published by Chinese authors and 2678 papers published by Indian authors in the field of computer science during 1971-2000 indicates that India's output is significantly higher than the Chinese output. However, China is catching up fast. Chinese researchers prefer to publish their research results in domestic journals, while Indian researchers prefer to publish their research results in journals published in the advanced countries of the West. Also the share of papers in journals covered by SCI for India was higher than from China. However, no significant difference has been observed in the impact of the research output of the two countries as seen by different impact indicators. Team research is more common in India as compared to China.     

Research productivity in education and psychology in the Philippines and comparison with ASEAN countries

An objective assessment using bibliometric indicators of research productivity in education and psychology in the Philippines was conducted. Results were then benchmarked against its Southeast Asian neighbors’ research productivity in the same fields. Results showed that the Philippines ranked low in research productivity compared to Singapore, Thailand, and Malaysia, particularly starting in the 1990s. Only a few researchers, mainly coming from a small number of higher education institutions, were publishing papers on a regular basis in a small range of journals. Those journals had either no or low impact factors and most papers had low citation counts. It also collaborated less with domestic and international institutions. This low research productivity was explained in terms of economic indicators, the local orientation of many social science research studies, funding, individual characteristics of researchers, and the epistemic culture of knowledge production in the country. However, the reforms initiated by the government, particularly in the higher education sector, would hopefully lead to a better research landscape and, consequently, improved research productivity in the near future.     

Sixty-four years of informetrics research: productivity,impact and collaboration

This paper analyses the information science research field of informetrics to identify publication strategies that have been important for its successful researchers. The study uses a micro-analysis of informetrics researchers from 5,417 informetrics papers published in 7 core informetrics journals during 1948–2012. The most productive informetrics researchers were analysed in terms of productivity, citation impact, and co-authorship. The 30 most productive informetrics researchers of all time span several generations and seem to be usually the primary authors of their research, highly collaborative, affiliated with one institution at a time, and often affiliated with a few core European centres. Their research usually has a high total citation impact but not the highest citation impact per paper. Perhaps surprisingly, the US does not seem to be good at producing highly productive researchers but is successful at producing high impact researchers. Although there are exceptions to all of the patterns found, researchers wishing to have the best chance of being part of the next generation of highly productive informetricians may wish to emulate some of these characteristics.     

An investigation into diabetes researcher’s perceptions of the Journal Impact Factor — reconsidering evaluating research

Currently the Journal Impact Factors (JIF) attracts considerable attention as components in the evaluation of the quality of research in and between institutions. This paper reports on a questionnaire study of the publishing behaviour and researchers preferences for seeking new knowledge information and the possible influence of JIF on these variables. 54 Danish medical researchers active in the field of Diabetes research took part. We asked the researchers to prioritise a series of scientific journals with respect to which journals they prefer for publishing research and gaining new knowledge. In addition we requested the researchers to indicate whether or not the JIF of the prioritised journals has had any influence on these decisions. Furthermore we explored the perception of the researchers as to what degree the JIF could be considered a reliable, stable or objective measure for determining the scientific quality of journals. Moreover we asked the researchers to judge the applicability of JIF as a measure for doing research evaluations. One remarkable result is that app. 80% of the researchers share the opinion that JIF does indeed have an influence on which journals they would prefer for publishing. As such we found a statistically significant correlation between how the researchers ranked the journals and the JIF of the ranked journals. Another notable result is that no significant correlation exists between journals where the researchers actually have published papers and journals in which they would prefer to publish in the future measured by JIF. This could be taken as an indicator for the actual motivational influence on the publication behaviour of the researchers. That is, the impact factor actually works in our case. It seems that the researchers find it fair and reliable to use the Journal Impact Factor for research evaluation purposes.     

The value of an overseas research trip

As an essential part of the academic environment, international scientific mobility draws considerable attention from researchers. Previous studies have indicated a strong relationship between scientific mobility and scientific output. However, few researchers have addressed the causality between them. The research questions in this study focused on how the international scientific mobilization of the researchers affects their number of international collaborations, their ability to get published at higher impact factor journals, the number of citations that they get. Based on the SCOPUS database of English language scientific journal articles, this paper revealed the causal effects of international scientific mobility of the researchers on their scientific productivity, collaborations, and impact on science using the synthetic control method. The author’s affiliation on their articles provided the geographical location that can be tracked in time to infer the international scientific mobility of each author. A sample of more than 79,000 immobile scientists was used to create the synthetic versions of over 1500 internationally mobile scientists, so that, the synthetic version of each mobile author best resembled the academic ability of her/his counterpart mobile author in the pre-mobilization period. This allowed investigating the effects of the international mobilization on their publications by comparing the post-mobilization publication characteristics of the mobile authors and their immobile synthetic controls.The findings show strong evidence of a substantial positive effect of scientific mobility on the ability to get published in more prestigious journals, the number of citations received in total and from overseas, and international collaborations. The magnitude of the effect is conditional on the duration of scientific mobility.

     

The evolution of the sleep science literature over 30 years: A bibliometric analysis

During the 1974–2004 period, the sleep literature had quadrupled (2384 publications in 1974, and 9721 in 2004) while overall scientific productivity had only doubled. The set of the seven most productive countries (USA, Japan, United Kingdom, Germany, France, Canada and Italy) in sleep research, and the geographical region distribution remained stable over the three decades. On the other hand several indicators appeared in the sleep research literature during the 1990s: the increasing productivity of sleep researchers; the growing number of countries publishing on sleep; the continuous creation of sleep-focused journals; the scattering of sleep publication among increasingly more scientific journals; the turnover among the leading journals; and the emergence of new entities such as China, Turkey, and the European Union.     

Comparison of research team activity in two biomedical fields

A study of the structure and scientific activity of the most productive Spanish research teams in two biomedical subfields, Pharmacology & Pharmacy and Cardiovascular System (SCI), during the period 1990–93 was carried out through bibliometric indicators. The teams were characterized according to their size, production, productivity, research level and expected impact factor of their output, collaboration pattern and interdisciplinarity. Main differences between both subfields were analyzed and explained by their different clinical/basic character. The study was undertaken to identify structural or dynamic features of teams associated with good scientific performance.     

Scholarly impact assessment: a survey of citation weighting solutions

Scientometrics - Scholarly impact assessment has always been a hot issue. It has played an important role in evaluating researchers, scientific papers, scientific teams, and institutions within...     

Impact of bibliometric studies on the publication behaviour of authors

It has been widely discussed how individuals change the way they act and react in studies just because they are under observation. In this paper, we try to analyse how this so-called Hawthorne effect applies to researchers that are the subject of bibliometric investigations. This encompasses individual assessments as well as international performance comparisons. We test various bibliometric indicators for notable changes in the last decade from a world-wide perspective and deduce explanations for changes from the observations. We then concentrate on the behaviour of German authors in particular, to show national trends. The German publication behaviour is evaluated in regard to citation rates and collaborations in publications and size, publisher country and impact of the journals chosen for publication. We can conclude that authors adapt their publication behaviour to aim for journals that are more internationally known and have a US publisher. Also, a trend from more specialized journals to journals with a broader scope can be observed that raises the question whether the implicit penalization of specialized fields in the bibliometrics leads to undesired shifts in conducted research.     

Highly cited research papers and the evaluation of a research university: A case study: Peking University 1974–2003

The academic level and scientific reputation is the most important merit of a research university. Publication of the scientific achievement in the world leading scientific journals is the key to asses a university's overall performance. Peking University is a leading university among the Chinese research universities, and the number of papers published in Science Citation Index (SCI) indexed journals has been on the top of the national list. In this paper, based on our long-term experience and practice in scientific management, we use scientometrics and informetrics method to analyze the academic performance of the researchers, departments and schools of Peking University, mainly using the citations of publications. Highly cited papers are specially important to the reputation of our university. We compare those data with some selected world well-known universities, hence, some important information can be deduced for the policy decision of the university. The results presented here is not only an academic survey, but also a guideline for the future strategic development of Peking University. This revised version was published online in June 2006 with corrections to the Cover Date.     

Does a country’s scientific ‘productivity’ depend critically on the number of country journals indexed?

In this paper, we examine the question whether it is meaningful to talk about the scientific productivity of nations based on indexes like the Science Citation Index or Scopus, when the journal set covered by them keeps changing with time. We hypothesize from the illustrative case of India’s declining productivity in the 1980s which correlated with a fall in its journals indexed, that an apparent increase/decrease in productivity for any country, based on observed change in its share of papers could, in fact, be an effect resulting from the inclusion of more/less journals from the country. To verify our hypothesis we have used SCIMAGO data. We found that for a set of 90 countries, the share of journals regressed on the share of papers gave a linear relationship that explained 80% of the variance. However, we also show that in the case of China’s unusual rise in world scientific productivity (to second rank crossing several other countries), there is yet another factor that needs to be taken into account. We define a new indicator—the JOURNAL PACKING DENSITY (JPD) or average number of papers in journals from a given country. We show that the packing density of Chinese journals has steadily increased over the last few years. Currently, Chinese journals have the highest ‘packing density’ in the world, almost twice the world average which is about 100 papers per journal per annum. The deviation of the JPD from the world average is another indicator which will affect so called ‘national productivities’ in addition to the number of national journals indexed. We conclude that in the context of a five fold increase in the number of journals indexed over 20 years, the simplistic notion of ‘scientific productivity’ as equivalent to papers indexed needs to be re-examined.     

Quantity versus impact of software engineering papers: a quantitative study

According to the data from the Scopus publication database, as analyzed in several recent studies, more than 70,000 papers have been published in the area of Software Engineering (SE) since late 1960’s. According to our recent work, 43% of those papers have received no citations at all. Since citations are the most commonly used metric for measuring research (academic) impact, these figures raise questions (doubts) about the (non-existing) impact of such a large set of papers. It is a reality that typical academic reward systems encourage researchers to publish more papers and do not place a major emphasis on research impact. To shed light on the issue of volume (quantity) versus citation-based impact of SE research papers, we conduct and report in this paper a quantitative bibliometrics assessment in four aspects: (1) quantity versus impact of different paper types (e.g., conference versus journal papers), (2) ratios of uncited (non-impactful) papers, (3) quantity versus impact of papers originating from different countries, and (4) quantity versus impact of papers by each of the top-10 authors (in terms of number of papers). To achieve the above objective, we conducted a quantitative exploratory bibliometrics assessment, comprised of four research questions, to assess quantity versus impact of SE papers with respect to the aspects discussed above. We extracted the data through a systematic, automated and repeatable process from the Scopus paper database, which we also used in two previous papers. Our results show that the distribution of SE publications has a major inequality in terms of impact overall, and also when categorized in terms of the above four aspects. The situation in the SE literature is similar to the other areas of science as studied by previous bibliometrics studies. Also, among our results is the fact that journal articles and conference papers have been cited 12.6 and 3.6 times on average, confirming the expectation that journal articles have more impact, in general, than conference papers. Also, papers originated from English-speaking countries have in general more visibility and impact (and consequently citations) when compared to papers originated from non-English-speaking countries. Our results have implications for improvement of academic reward systems, which nowadays mainly encourage researchers to publish more papers and usually neglect research impact. Also, our results can help researchers in non-English-speaking countries to consider improvements to increase their research impact of their upcoming papers.     

Inequality and collaboration patterns in Canadian nanotechnology: implications for pro-poor and gender-inclusive policy

Policymakers and scholars are increasingly concerned with how nanotechnology can reduce inequalities and provide benefits for disadvantaged and poor communities. This paper simultaneously addresses two concerns related to nanotechnology and equity: the lack of research and development focused on nanotechnology applications that benefit developing nations (pro-poor R&D) and the lack of women in nanotechnology fields. The paper focuses on Canada, an affluent country committed to both pro-poor and gender responsive policies. Social network analysis is used to examine the relationship between gender and collaboration patterns of authors and inventors whose work is related to pro-poor applications of nanotechnology. Findings reveal that female first-authored papers have a lower citation rate and are published in higher ranked journals compared to those papers first-authored by men. Nevertheless, when women are last or corresponding authors, their papers receive equal or higher citation rates and are published in lower or similar ranked journals. Women are as, or more, collaborative as their male peers in their co-authorship and co-inventorship networks. While the majority of male authors and male inventors collaborate exclusively with men, those involved in a mixed-gender team outperform male-only teams. Women, as both authors and inventors, are involved in more gender-balanced collaboration teams. The study calls for development and implementation of gender-related policies in Canada to increase the prevalence of female scientists in collaboration networks, and to support the participation of women in pro-poor areas.     

An attempt of surveying and classifying bibliometric indicators for scientometric purposes

An attempt is made for the survey and classification of bibliometric indicators applicable for assessment of publication performance of researchers active in natural sciences. Indicators can be classified as publication and citation ones which may refer to impact and quantity of publication activity of researcher(s), teams, institutes or countries. Taking into account the possible reference standards, the indicators are classified as simple, specific, balance, distribution and relative ones. In order to evaluate publication activity both qualitatively and quantitatively, relative citation indicators can be recommended, which relate citations received to the sum of impact factors of the journals, where the papers were published or give the relative measure of the average citedness of papers related to that of papers in journals in a similar subfield.