ResearchGate and Google Scholar: how much do they differ in publications,citations and different metrics and why?

ResearchGate has emerged as a popular professional network for scientists and researchers in a very short span. Similar to Google Scholar, the ResearchGate indexing uses an automatic crawling algorithm that extracts bibliographic data, citations, and other information about scholarly articles from various sources. However, it has been observed that the two platforms often show different publication and citation data for the same institutions, journals, and authors. While several previous studies analysed different aspects of ResearchGate and Google Scholar, the quantum of differences in publications, citations, and metrics between the two and the probable reasons for the same are not explored much. This article, therefore, attempts to bridge this research gap by analysing and measuring the differences in publications, citations, and different metrics of the two platforms for a large data set of highly cited authors. The results indicate that there are significantly high differences in publications and citations for the same authors captured by the two platforms, with Google Scholar having higher counts for a vast majority of the cases. The different metrics computed by the two platforms also differ in their values, showing different degrees of correlation. The coverage policy, indexing errors, author attribution mechanism, and strategy to deal with predatory publishing are found to be the main probable reasons for the differences in the two platforms.

     

$$C^3$$-index: a PageRank based multi-faceted metric for authors’ performance measurement

Ranking scientific authors is an important but challenging task, mostly due to the dynamic nature of the evolving scientific publications. The basic indicators of an author’s productivity and impact are still the number of publications and the citation count (leading to the popular metrics such as h-index, g-index etc.). H-index and its popular variants are mostly effective in ranking highly-cited authors, thus fail to resolve ties while ranking medium-cited and low-cited authors who are majority in number. Therefore, these metrics are inefficient to predict the ability of promising young researchers at the beginning of their career. In this paper, we propose \(C^3\)-index that combines the effect of citations and collaborations of an author in a systematic way using a weighted multi-layered network to rank authors. We conduct our experiments on a massive publication dataset of Computer Science and show that—(1) \(C^3\)-index is consistent over time, which is one of the fundamental characteristics of a ranking metric, (2) \(C^3\)-index is as efficient as h-index and its variants to rank highly-cited authors, (3) \(C^3\)-index can act as a conflict resolution metric to break ties in the ranking of medium-cited and low-cited authors, (4) \(C^3\)-index can also be used to predict future achievers at the early stage of their career.     

Using Hypertext Metrics to Measure Research Output Levels

Two common ways to measure the "output" of a researcher (or research group) are to count numbers of publications and to count the citations (references to these publications in publications of others). These simple methods are flawed because they cannot easily take into account the differences in publication and citation habits in different scientific communities.An alternative approach is to view citations as hypertext links, and to use or adapt hypertext metrics to compare the scientific output of researchers, in comparison to that of others in areas with similar publication and citation patterns. We show how hypertext metrics, introduced by Botafogo, Rivlin and Shneiderman, can be modified in order to identify comparable research fields based on their publication and citation pattern. An author's performance can then be compared to that of others in research fields with a similar pattern.     

Measuring recent research performance for Chinese universities using bibliometric methods

This paper focuses on measuring the academic research performance of Chinese universities by using Scopus database from 2007 to 2010. We have provided meaningful indicators to measure the research performance of Chinese universities as compared to world class universities of the US and the European region. Using these indicators, we first measure the quantity and quality of the research outcomes of the universities and then examine the internationalization of research by using international collaborations, international citations and international impact metrics. Using all of this data, we finally present an overall score called research performance point to measure the comprehensive research strength of the universities for the selected subject categories. The comparison identifies the gap between Chinese universities and top-tier universities from selected regions across various subject areas. We find that Chinese universities are doing well in terms of publication volume but receive less citations from their published work. We also find that the Chinese universities have relative low percentage of publications at high impact venues, which may be the reason that they are not receiving more citations. Therefore, a careful selection of publication venues may help the Chinese universities to compete with world class universities and increase their research internationalization.     

Author name disambiguation using a graph model with node splitting and merging based on bibliographic information

Author ambiguity mainly arises when several different authors express their names in the same way, generally known as the namesake problem, and also when the name of an author is expressed in many different ways, referred to as the heteronymous name problem. These author ambiguity problems have long been an obstacle to efficient information retrieval in digital libraries, causing incorrect identification of authors and impeding correct classification of their publications. It is a nontrivial task to distinguish those authors, especially when there is very limited information about them. In this paper, we propose a graph based approach to author name disambiguation, where a graph model is constructed using the co-author relations, and author ambiguity is resolved by graph operations such as vertex (or node) splitting and merging based on the co-authorship. In our framework, called a Graph Framework for Author Disambiguation (GFAD), the namesake problem is solved by splitting an author vertex involved in multiple cycles of co-authorship, and the heteronymous name problem is handled by merging multiple author vertices having similar names if those vertices are connected to a common vertex. Experiments were carried out with the real DBLP and Arnetminer collections and the performance of GFAD is compared with three representative unsupervised author name disambiguation systems. We confirm that GFAD shows better overall performance from the perspective of representative evaluation metrics. An additional contribution is that we released the refined DBLP collection to the public to facilitate organizing a performance benchmark for future systems on author disambiguation.     

Citation pattern and lifespan: a comparison of discipline,institution, and individual

Citation frequency is often used in hiring and tenure decisions as an indicator of the quality of a researcher’s publications. In this paper, we examine the influence of discipline, institution, journal impact factor, length of article, number of authors, seniority of author, and gender on citation rate of top-cited papers for academic faculty in geography and forestry departments. Self-citation practices and patterns of citation frequency across post-publication lifespan were also examined. Citation rates of the most-highly cited paper for all tenured forestry (N = 122) and geography (N = 91) faculty at Auburn University, Michigan State University, Northern Arizona University, Oklahoma State University, Pennsylvania State University, Texas A&M University, University of Florida, University of Massachusetts, University of Washington, and Virginia Tech were compared. Foresters received significantly more citations than geographers (t = 2.46, P = 0.02) and more senior authors received more citations than junior researchers (r ² = 0.14, P = 0.03). Articles published in journals with higher impact factors also received more citations (r ² = 0.28, P = 0.00). The median self-citation rate was 10% and there was no temporal pattern to the frequency of citations received by an individual article (x ² = 176). Our results stress the importance of only comparing citation rates within a given discipline and confirm the importance of author-seniority and journal rankings as factors that influence citation rate of a given article.     

Classic articles on social work field in Social Science Citation Index: a bibliometric analysis

This study aimed to identify and analyze characteristics of classic articles published in the Web of Science social work subject category from 1856 to 2011. Articles that have been cited at least 50 times were assessed regarding publication outputs, distribution of outputs in journals, publications of authors, institutions, countries as well as citation life cycles of articles with the highest total citations since its publication up to 2011 and the highest citations in 2011. Five bibliometric indicators were used to evaluate source countries, institutions, and authors. Results showed that 721 of the most highly referenced articles, published between 1957 and 2008, had been cited at least 50 times. Child Abuse & Neglect and American Journal of Community Psychology published the most classic articles. USA produced 89 % of classic articles and also published the most number of single, internationally collaborative, first author, and corresponding author classic articles. The top 38 productive institutions were all located in the US. The University of Illinois was the most productive institution for the total classic articles while University of California, Los Angeles produced the most inter-institutionally collaborative articles and Arizona State University published the most single institution articles. Furthermore, a new indicator, Y-index was successfully applied to evaluate publication characteristics of authors and institutions. High percentage of authors had the same numbers of first author and corresponding author status of classic articles in social work field.     

Evaluation of <Emphasis Type="Italic">h</Emphasis>-index,its variants and extensions based on publication age &amp; citation intensity in civil engineering

The scientific community has proposed diversified set of parameters to rank researchers, including publications, citations, h-index, different variants and extensions of h-index. However, there is a debate in the scientific Community which index ranks authors in a better way. Current state-of-the-art depicts that these indices are evaluated on imaginary case scenarios and small datasets. Furthermore, these indices are evaluated on different datasets, making it difficult to grasp the contribution and importance of each index over the others. To analyze the individual behavior of each index, these indices should comprehensively be evaluated on some extensive data set. This study emphasizes on the scrutiny of h-index, some of its variants and extensions to rank authors. These indices are evaluated using a comprehensive data set of Civil Engineering field. For the evaluation of results obtained from these indices, first correlation was calculated among indices. There exists weak correlation between various indices, which demonstrates that the author’s rankings acquired from these indices are not identical. Secondly, occurrences of awardees are checked in all ranked lists. The prestigious award winners of four Civil Engineering societies are considered as a benchmark. In top 10% of ranked list, maximum 47% of the awardees were brought by Wu-index. Overall, none of the index succeeded in bringing 100% awardees to the top rankings. Highest number of awardees on top of all ranked lists are found to be from ACI (American Concrete Institute), which shows ACI might be dependent on these indices for its criterion to honor awards.     

Counting methods are decisive for rankings based on publication and citation studies

Summary For all rankings of countries research output based on number of publications or citations compared with population, GDP, R&D and public R&D expenses, and other national characteristics the counting method is decisive. Total counting (full credit to a country when at least one of the authors is from this country) and Fractional Counting (a country receives a fraction of full credit for a publication equal to the fraction of authors from this country) of publications give widely different results. Counting methods must be stated, rankings based on different counting methods cannot be compared, and Fractional Counting is to be preferred.     

#Psychology: a bibliometric analysis of psychological literature in the online media

Online media and especially social media are becoming more and more relevant to our everyday life. Reflecting this tendency in the scientific community, alternative metrics for measuring scholarly impact on the web are increasingly proposed, extending (or even replacing) traditional metrics (e.g., citations, journal impact factor, etc.). This paper explores the relationship between traditional metrics and alternative metrics for psychological research in the years from 2010 to 2012. Traditional publication metrics (e.g., number of citations, impact factor) and alternative metrics (collected from Altmetric, a website that collects and counts references as they appear in Wikipedia, public policy documents, research blogs, mainstream media, or social networks) were extracted and compared, using a dataset of over 245,000 publications from the Web of Science. Results show positive, small to medium, correlations on the level of individual publications, and frequently medium to high correlations on the level of research fields of Psychology. The more accumulated the level of analysis, the higher the correlations. These findings are fairly robust over time and comparable to findings from research areas other than Psychology. Additionally, a new metric, the Score Factor, is proposed as a useful alternative metric to assess a journal’s impact in the online media.     

Bibliometric analysis of biotechnology research in Pakistan

In this study we present an analysis of the research trends in Pakistan in the field of biotechnology for the period 1980–2011. Starting with just 15 publications in 1980 with a negligible annual growth rate for the initial 15 years, the number of publications reached 3,273 in 2011 with an annual growth rate of 22 % for the last 15 years. This growth in publications is studied through factors such as Relative Growth Rate and Doubling Time. A comparison of organizations actively engaged in research in biotechnology is made through factors such as their total publications, total citations, and average citations per paper and indices that determine the quality of publications like h-index, g-index, hg-index and p-index. University of Karachi shows the highest number of publications (2,698), while National Institute of Biotechnology and Genetic Engineering with fewer publications shows the highest average citation per paper (8.07). Agha Khan University however, shows the highest h, g, hg and p indices.     

How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications

In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we collected metrics for 20,000 random publications from the Web of Science. We studied both the presence and distribution of altmetrics in the set of publications, across fields, document types and over publication years, as well as the extent to which altmetrics correlate with citation indicators. The main result of the study is that the altmetrics source that provides the most metrics is Mendeley, with metrics on readerships for 62.6 % of all the publications studied, other sources only provide marginal information. In terms of relation with citations, a moderate spearman correlation (r = 0.49) has been found between Mendeley readership counts and citation indicators. Other possibilities and limitations of these indicators are discussed and future research lines are outlined.     

Evolution of the publications in clinical neurology: scientific impact of different countries during the 2000–2009 period

We analyzed the productivity and visibility of publications on the subject category of Clinical Neurology by countries in the period 2000–2009. We used the Science Citation Index Expanded database of the ISI Web of Knowledge. The analysis was restricted to the citable documents. Bibliometric indicators included the number of publications, the number of citations, the median and interquartile range of the citations, and the h-index. We identified 170,483 publications (84.9 % original articles) with a relative increase of 28.5 % throughout the decade. Fourteen countries published over 2,000 documents in the decade and received more than 50,000 citations. The average of citations received per publication was 8 (interquartile range: 3–20) and the h-index was 261. USA was the country with the highest number of publications, followed by Germany, Japan, the UK and Italy. Moreover, USA publications had the largest number of citations received (44.5 % of total), followed by the UK, Germany, Canada, and Italy. On the other hand, Sweden, the Netherlands and the UK had the highest median citations for their total publications. During the period 2000–2009 there was a significant increase in Clinical Neurology publications. Most of the publications and citations comprised 14 countries, with the USA in the first position. Interestingly, most of the publications and citations originated from only 14 countries, with European countries with relatively low population, such as Switzerland, Austria, Sweden, Belgium, and the Netherlands, in this top group.     

Impact evaluation of the voluntary early retirement policy on research and technology outputs of the faculties of science in Morocco

Scientometric indicators or science metrics, conventional and derived ones, are used in ex-post evaluating of a government policy with impact on research system. Publications, citations, h-index, Glänzel model, and patents are applied in both micro and meso levels. This provides useful insight into the impact of the voluntary early retirement policy on research and technological outputs of the faculties of science in Morocco and consequently on the overall Morocco??s research system. The use of these metrics showed that the effect of the initiative was quite limited by affecting an average of 8% of the professor staffs of these institutions. Furthermore, each professor benefiting from this initiative had produced an average of 3.7 publications indexed in SCI in all his (her) career. The few number of the publications attributed to these professors had been gradually decreasing even 6 years before the initiative. No specific scientific field had intensively been struck. The findings also support that these professors were in general more ??author?? than ??inventor??. Inventor-professor institutions were likely more affected by the initiative. By means of these metrics, even if the initiative had not contributed to rejuvenate the professor-staffs of the faculties of science in Morocco, would nevertheless be a stimulus of their research system with respect to their scientometric indicators.     

Does cumulative advantage affect collective learning in science? An agent-based simulation

Agent-based simulation can model simple micro-level mechanisms capable of generating macro-level patterns, such as frequency distributions and network structures found in bibliometric data. Agent-based simulations of organisational learning have provided analogies for collective problem solving by boundedly rational agents employing heuristics. This paper brings these two areas together in one model of knowledge seeking through scientific publication. It describes a computer simulation in which academic papers are generated with authors, references, contents, and an extrinsic value, and must pass through peer review to become published. We demonstrate that the model can fit bibliometric data for a token journal, Research Policy. Different practices for generating authors and references produce different distributions of papers per author and citations per paper, including the scale-free distributions typical of cumulative advantage processes. We also demonstrate the model’s ability to simulate collective learning or problem solving, for which we use Kauffman’s NK fitness landscape. The model provides evidence that those practices leading to cumulative advantage in citations, that is, papers with many citations becoming even more cited, do not improve scientists’ ability to find good solutions to scientific problems, compared to those practices that ignore past citations. By contrast, what does make a difference is referring only to publications that have successfully passed peer review. Citation practice is one of many issues that a simulation model of science can address when the data-rich literature on scientometrics is connected to the analogy-rich literature on organisations and heuristic search.     

Understanding the formation of interdisciplinary research from the perspective of keyword evolution: a case study on joint attention

We measure the knowledge flows between countries by analysing publication and citation data, arguing that not all citations are equally important. Therefore, in contrast to existing techniques that utilize absolute citation counts to quantify knowledge flows between different entities, our model employs a citation context analysis technique, using a machine-learning approach to distinguish between important and non-important citations. We use 14 novel features (including context-based, cue words-based and text-based) to train a Support Vector Machine (SVM) and Random Forest classifier on an annotated dataset of 20,527 publications downloaded from the Association for Computational Linguistics anthology (http://allenai.org/data.html). Our machine-learning models outperform existing state-of-the-art citation context approaches, with the SVM model reaching up to 61% and the Random Forest model up to a very encouraging 90% Precision–Recall Area Under the Curve, with 10-fold cross-validation. Finally, we present a case study to explain our deployed method for datasets of PLoS ONE full-text publications in the field of Computer and Information Sciences. Our results show that a significant volume of knowledge flows from the United States, based on important citations, are consumed by the international scientific community. Of the total knowledge flow from China, we find a relatively smaller proportion (only 4.11%) falling into the category of knowledge flow based on important citations, while The Netherlands and Germany show the highest proportions of knowledge flows based on important citations, at 9.06 and 7.35% respectively. Among the institutions, interestingly, the findings show that at the University of Malaya more than 10% of the knowledge produced falls into the category of important. We believe that such analyses are helpful to understand the dynamics of the relevant knowledge flows across nations and institutions.     

Research productivity among trainee anaesthetists in Ireland: a cross-sectional study

Objective measures of research performance are necessary to facilitate academic advancement of trainee physicians. In this cross-sectional study, all anaesthetists (n?=?98) in higher specialist training in Ireland were surveyed to determine bibliometrics of their scientific publications and individual and institutional characteristics that can influence research productivity. For trainees with publications, the median (range) h-index was 1 (0?C4). There was a positive correlation between participation in a formal research program and increased research productivity using mean citations per publication (r ²?=?0.26, P?=?0.006) and h-index (r ²?=?0.26, P?=?0.006). There was a positive correlation between formal mentorship and mean citations per publication (r ²?=?0.15, P?=?0.04) and h-index (r ²?=?0.17, P?=?0.03).     

International earth science literature from Turkey — 1970–2005: Trends and possible causes

We investigated the publication trends in the international earth science literature coming out of Turkey in the period of 1970–2005 using the Science Citation Index Expanded database. A database of 2310 earth science publications with at least one of the authors with an address in Turkey was compiled. The number of earth science publications from Turkey shows a very rapid increase starting in the 1990’s in parallel with the increase in the total scientific output of Turkey. In the last decade the annual growth rate has been 16%. There was also a concomitant increase in the number of citations. The causes of the sharp increase in the publication numbers are, in order of importance, changes in the rules of academic promotion and appointment, changes in academic attitudes towards publishing, increasing support for research, financial incentives for publishing, and expansion of higher education. However, the sharp increase in the publication numbers was not accompanied by a similar increase in the impact of the publications as measured by the citations. Although publications with first authors from outside Turkey make up only 20% of the Turkish earth science publications in the period 1970–2005, these account for 38% of the total citations, and constitute 48 out of 100 most cited papers.     

Journal impact evaluation: a webometric perspective

In theory, the web has the potential to provide information about the wider impact of academic research, beyond traditional scholarly impact. This is because the web can reflect non-scholarly uses of research, such as in online government documents, press coverage or public discussions. Nevertheless, there are practical problems with creating metrics for journals based on web data: principally that most such metrics should be easy for journal editors or publishers to manipulate. Nevertheless, two alternatives seem to have both promise and value: citations derived from digitised books and download counts for journals within specific delivery platforms.     

Publication trends in materials science: a global perspective

This paper attempts to highlight quantitatively and qualitatively the growth and development of world literature on materials science in terms of publication output and citations as per Web of Science (2006–2010). The objective of the study was to perform a scientometric analysis of all materials science research publications in the world. The parameters studied include growth of publications and citations, continent-wise distribution of publications and citations, country-wise distribution of publications, domain-wise distribution of publications and citations, publication efficiency index, distribution of publications and citations according to number of collaborating countries, variation of mean impact factor in materials science domains, identification of highly cited publications and highly preferred journals, quality of research output and application of Bradford’s law.