Learning about learning: patterns of sharing of research knowledge among Education,Border, and Cognitive Science fields

This study explores the patterns of exchange of research knowledge among Education Research, Cognitive Science, and what we call “Border Fields.” We analyze a set of 32,121 articles from 177 selected journals, drawn from five sample years between 1994 and 2014. We profile the references that those articles cite, and the papers that cite them. We characterize connections among the fields in sources indexed by Web of Science (WoS) (e.g., peer-reviewed journal articles and proceedings), and connections in sources that are not (e.g., conference talks, chapters, books, and reports). We note five findings—first, over time the percentage of Education Research papers that extensively cite Cognitive Science has increased, but the reverse is not true. Second, a high percentage of Border Field papers extensively cite and are cited by the other fields. Border Field authors’ most cited papers overlap those most cited by Education Research and Cognitive Science. There are fewer commonalities between Educational research and Cognitive Science most cited papers. This is consistent with Border Fields being a bridge between fields. Third, over time the Border Fields have moved closer to Education Research than to Cognitive Science, and their publications increasingly cite, and are cited by, other Border Field publications. Fourth, Education Research is especially strongly represented in the literature published outside those WoS-indexed publications. Fifth, the rough patterns observed among these three fields when using a more restricted dataset drawn from the WoS are similar to those observed with the dataset lying outside the WoS, but Education Research shows a far heavier influence than would be indicated by looking at WoS records alone.

     

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

     

A quantitative analysis of determinants of non-citation using a panel data model

The term “non-citation factor” refers to the percentage of never-cited papers in a citation time window, a common phenomenon in the science world. Some scholars have qualitatively explored the reasons for not citing a publication, and quantitatively analyzed the mathematical functional relations between the “non-citation factor” and “impact factor of a journal.” This study simultaneously considers the mutual relations and closeness degree between the “non-citation factor” and different influencing factors from a novel perspective—that of using a more structuralized panel data model. The analysis revealed that the determinants, including “impact factor of journal,” “age of journal,” “average number of references per paper in journal,” and “issues of journal,” exerted an extremely small but positive influence (<?0.025) on the decline of “percentage of never-cited papers in the citation time window of publication year or 3 years.” That means the improvement of these determinants can decrease the percentage of never-cited papers. The “impact factor of the journal” always had the biggest positive influence, while the “average number of references per paper in journal” always had the smallest positive influence. In wider citation time windows of 3 or 6 years, two determinants—“number of publications in journal” and “amount of interdisciplinarity in journal”—began to exert a negative effect with a positive correlation coefficient on the decline of the “non-citation factor.” That means the improvement of these two determinants cannot decrease the value of the “non-citation factor,” even though they can increase its value. It is worth noting that the “impact factor of the journal” had a positive influence on the decline of the percentages of never-cited papers in the citation time window of publication year or 3 years, and began to play a negative role in the decline of percentage of never-cited papers in the citation time window of 6 years. Finally, three variables—“average number of authors per paper in journal,” “average number of references per paper in journal,” and “issues of journal”—no longer exerted an influence on the decline of percentages of never-cited papers in the citation time window of 6 years, while “age of journal” and “average number of pages per paper in journal” still made a positive contribution. Our findings could help research institutions, researchers, editors, and publishers understand the positively or negatively influential factors that lead to non-citation, thus improving the chance of papers being cited and having some academic influence.     

How much do different ways of calculating percentiles influence the derived performance indicators? A case study

Bibliometric indicators can be determined by comparing specific citation records with the percentiles of a reference set. However, there exists an ambiguity in the computation of percentiles because usually a significant number of papers with the same citation count are found at the border between percentile rank classes. The present case study of the citations to the journal Europhysics Letters (EPL) in comparison with all physics papers from the Web of Science shows the deviations which occur due to the different ways of treating the tied papers in the evaluation of the percentage of highly cited publications. A strong bias can occur, if the papers tied at the threshold number of citations are all considered as highly cited or all considered as not highly cited.     

Assessing non-standard article impact using F1000 labels

Faculty of 1000 (F1000) is a post-publishing peer review web site where experts evaluate and rate biomedical publications. F1000 reviewers also assign labels to each paper from a standard list or article types. This research examines the relationship between article types, citation counts and F1000 article factors (FFa). For this purpose, a random sample of F1000 medical articles from the years 2007 and 2008 were studied. In seven out of the nine cases, there were no significant differences between the article types in terms of citation counts and FFa scores. Nevertheless, citation counts and FFa scores were significantly different for two article types: “New finding” and “Changes clinical practice”: FFa scores value the appropriateness of medical research for clinical practice and “New finding” articles are more highly cited. It seems that highlighting key features of medical articles alongside ratings by Faculty members of F1000 could help to reveal the hidden value of some medical papers.     

Evaluating scientific impact of publications: combining citation polarity and purpose

Citation counts are commonly used to evaluate the scientific impact of a publication on the general premise that more citations probably mean more endorsements. However, two questionable assumptions underpin this idea: a) that all authors contributed equally to the paper; and b) that the endorsement is positive. Obviously, neither of these assumptions hold true. Hence, with this study, we examine two components of citations—their purpose, i.e., the reason for the citation, and polarity, being the author’s attitude toward the cited work. Our findings provide a new perspective on the scientific impact of highly-cited publications. Our methodology consists of three steps. Firstly, a pre-trained model composed of a Word2Vec—a well-known word embedding approach—and a convolutional neural network (CNN) is used to identify citation polarity and purpose. Secondly, in a set of highly-cited papers, we compare eight categories of purpose from foundational to critical and three categories of polarity: positive, negative, and neutral. We further explore how different types of papers—those discussing discoveries or those discussing utilitarian topics—influence the evaluation of scientific impact of papers. Finally, we mine and discover the knowledge (e.g. method, concept, tool or data) to explain the actual scientific impact of a highly-cited paper. To demonstrate how combining citation polarity with purpose can provide far greater details of a paper’s scientific impact, we undertake a case study with 370 highly-cited journal articles spanning “Biochemistry & Molecular Biology” and “Genetics & Heredity”. The results yield valuable insights into the assumption about citation counts as a metric for evaluating scientific impact.

     

Correlation between quality and quantity in scientific production: A case study of Iranian organizations from 1997 to 2006

In order to prevent the formation of a gap between the quality and quantity in Iranian scientific publications, this study makes an effort to analyze Iranian scientific publications indexed on the ISI Web of Science database using quantitative and qualitative scientometrics criteria over a ten year period. As a first step, all Iranian institutes were divided into three categories; universities, research institutes and other organizations. Then they were compared according to quantitative and qualitative criteria. Second, the correlation between the quality and quantity of the publications was measured. The research findings indicated that, according to qualitative criteria (citation, citation impact and percentage of cited documents) there are no meaningful differences among the three groups, while regarding quantitative criterion(number of papers), universities rank higher than the other two groups. The results also indicated that there is a positive and meaningful correlation among qualitative and quantitative criteria in the scholarly scientific publications conducted by Iranian organizations. In other words, in Iranian organizations the quality of publications increases as their quantity increases. The comparison of magnitude of correlation between these two criteria in the three categories reveals the fact that the correlation between number of papers and citations criterion in research institutes is stronger than the other two groups.     

A parameter-free index for identifying under-cited sleeping beauties in science

We developed a systematic methodology for identifying the under-cited (or not-so-highly cited) Sleeping Beauty (SB) publications and tried to figure out their key characteristics. Based on the identification framework of “beauty coefficient” (B) introduced by Ke et al. (Proc Natl Acad Sci USA 112:7426–7431, 2015), taking into account the whole citation history of the publications concerned, we substituted yearly citations in “beauty coefficient” with yearly cumulative percentage of citations, and eliminated the denominator in “beauty coefficient” since the curve of a given document’s cumulative citations is always monotonically increasing if only the document is cited. The value of the modified beauty coefficient is denoted as Bcp. We also redefined the awakening year, sleeping length and sleeping depth within the Bcp framework with the intention of avoiding arbitrary thresholds as much as possible. We tested the new index using the data of SB articles identified from Science and Nature. The results showed that Bcp is more sensitive in identifying the “lower level SBs”, which refers to the case when the total citations and the maximum annual citations of SBs are not so high in comparison with other typical SBs. Bcp works better than B in at least two aspects: (1) it “punishes” the situations when the SBs experienced early citations instead of continuous sleeping; (2) it allows for comparing the extent of delayed citation impact of publications in different fields with different citation patterns. We also figured out some key characteristics of such SB publications and pondered some policy implications about the associations of SB publications with transformative research, research front and research evaluation.     

Microwave effect: analyzing citations from classic theories and their reinventions—a case study from a classic paper in aquatic ecology—Brooks & Dodson, 1965

Citations play an essential role in creating a knowledge network and recognizing relevant contributions during the process of scientific production. Despite the citations establishing the links between new evidence and the preceding ideas, classic articles may not be cited adequately. Our aim is to identify if classic studies are cited over time and if the recent studies are producing new knowledge or just “giving a new look” to pre-existing ideas. We evaluated whether the theory proposed by Brooks and Dodson (Science 150(3692): 28–35, 1965)-Size-efficiency Hypothesis was referenced in studies on the subject since its publication. Through the analysis of 1480 scientific papers, we quantified—from 1965 to 2018—the citation index (CI) of the original article considering the number of articles produced on the topic per year and the number of citations to other authors (intermediaries). We observed that 60% of the papers and 59% of the intermediaries do not refer to the original article. The CI was low and negatively affected by the age of the original article, showing that the frequency of citation was lower than the rate by which articles on the topic were published. There is a tendency to cite more recent articles and articles that corroborate their own findings. Our data illustrated the microwave effect, in which pre-existing ideas and theories are “reheated” by more recent articles where little of the original idea is modified. The microwave effect can create the impression of scientific advancement when there is little being added to the knowledge already produced.

     

Influence of proceedings papers on citation impact in seven sub-fields of sustainable energy research 2005–2011

This paper analyses the following seven sub-fields of Sustainable Energy Research with respect to the influence of proceedings papers on citation patterns across citing and cited document types, overall sub-field and document type impacts and citedness: the Wind Power, Renewable Energy, Solar and Wave Energy, Geo-thermal, Bio-fuel and Bio-mass energy sub-fields. The analyses cover peer reviewed research and review articles as well as two kinds of proceeding papers from conferences published 2005–2009 in (a) book series or volumes and (b) special journal issues excluding meeting abstracts cited 2005–2011 through Web of Science. Central findings are: The distribution across document types of cited versus citing documents is highly asymmetric. Predominantly proceedings papers from both proceeding volumes as well as published in journals cite research articles (60–76 %). Largely, journal-based proceedings papers are cited rather than papers published in book series or volumes and have field impacts corresponding to research articles. With decreasing proceedings paper dominance in research fields the ratio of proceeding paper volumes over journal-based proceedings papers decreases significantly and the percentage of proceedings papers in journals citing journal-based proceedings papers over all publications citing journal-based proceedings papers decreases significantly (from 26.3 % in Wind Power to 4 % in Bio Fuel). Further, the segment of all kinds of proceedings papers (the combined proceedings paper types) citing all proceedings papers over all publications citing all kinds of proceedings papers decreases significantly (from 36.1 % in Wind Power to 11.3 % in Bio Fuel). Simultaneously the field citedness increases across the seven research fields. The distribution of citations from review articles shows that novel knowledge essentially derives directly from research articles (53–72 %)—to a much less extent from proceedings publications published in journals (9–13 %).     

Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis

During Eugene Garfield’s (EG’s) lengthy career as information scientist, he published about 1500 papers. In this study, we use the impressive oeuvre of EG to introduce a new type of bibliometric networks: keyword co-occurrences networks based on the context of citations, which are referenced in a certain paper set (here: the papers published by EG). The citation context is defined by the words which are located around a specific citation. We retrieved the citation context from Microsoft Academic. To interpret and compare the results of the new network type, we generated two further networks: co-occurrence networks which are based on title and abstract keywords from (1) EG’s papers and (2) the papers citing EG’s publications. The comparison of the three networks suggests that papers of EG and citation contexts of papers citing EG are semantically more closely related to each other than to titles and abstracts of papers citing EG. This result accords with the use of citations in research evaluation that is based on the premise that citations reflect the cognitive influence of the cited on the citing publication.     

Citations: Indicators of significance?

What makes a scientific article significant? This paper-part of a larger study which will examine how various kinds of significance can be related to one another in a coherent theoretical framework-focusses on the processes by which new knowledge claims are being integrated into the cognitive structure when they are cited in other papers. Citations appear both as “threads” linking the citing papers to the existing literature in the field, and as elements fulfilling specific functions within the arguments made in these papers. We have found that (1) it is misleading to equate every article with a single knowledge claim, let alone with an attempt to construct “a fact”; (2) even when the same “sentence” is cited repeatedly, it can be put to quite different uses in the citing papers; and (3) the process of codification of scientific knowledge through the use of references appears to be far more complex and multi-dimensional than citation context analyses focussing on the use and the gradual disappearance of modalities would lead us to believe. Some consequences for the use use of citation analysis to reconstruct cognitive structures will be discussed.     

Positive results receive more citations, but only in some disciplines

Negative results are commonly assumed to attract fewer readers and citations, which would explain why journals in most disciplines tend to publish too many positive and statistically significant findings. This study verified this assumption by counting the citation frequencies of papers that, having declared to “test” a hypothesis, reported a “positive” (full or partial) or a “negative” (null or negative) support. Controlling for various confounders, positive results were cited on average 32 % more often. The citation advantage, however, was unequally distributed across disciplines (classified as in the Essential Science Indicators database). Using Space Science as the reference category, the citation differential was positive and formally statistically significant only in Neuroscience & Behaviour, Molecular Biology & Genetics, Clinical Medicine, and Plant and Animal Science. Overall, the effect was significantly higher amongst applied disciplines, and in the biological compared to the physical and the social sciences. The citation differential was not a significant predictor of the actual frequency of positive results amongst the 20 broad disciplines considered. Although future studies should attempt more fine-grained assessments, these results suggest that publication bias may have different causes and require different solutions depending on the field considered.     

The Garfield impact factor, one of the fundamental indicators in scientometrics

The paper summarizes some basic features of the Garfield impact factor (GF). Accordingly, GF should be regarded as a scientometric indicator representing the relative contribution of journals to the total impact of information in a field. For calculating GF, both from theoretical and practical reasons the “ratio of the sums” method is recommended over the “mean of the ratios” method. Scientific advances are made by the most influential, presumably most frequently cited articles. The distribution of citations among the publications is skewed in journals. Consequently, the GF index will be influenced primarily by the highly cited papers. It follows, GF represents the most valuable part of the information in journals quantitatively, and even therefore it may be regarded as a reliable impact indicator.     

Geographical knowledge diffusion and spatial diversity citation rank

This paper proposes a citation rank based on spatial diversity (SDCR) in terms of cities and countries, focusing on the measurement of the “spatial” aspect in citation networks. Our main goal is to solve the citation bias caused by different geographical locations of citations. We empirically investigate spatial properties of citing distances, citation patterns and spatial diversity to understand geographical knowledge diffusion, based on the data from “Transportation Science and Technology” subject category in the Web of Science (1966–2009). We also compare the proposed ranking method with other bibliometric measures, and conduct a case study to figure out the recent ranks of the well-established authors in Transportation research. It is found that the SDCR of a focal author is highly correlated with the sum of spatial diversity weights (“strength”) of all his in-links, and it is better to set the damping factors smaller than 0.75 when ranking authors with various initial academic years by SDCR. The cases show that Hong Kong is becoming a cluster in Transportation research.     

Bibliometric analysis of complementary and alternative medicine research over three decades

This study applies bibliometric analysis to investigate the quantity and citation impact of scientific papers in the field of complementary and alternative medicine (CAM). The data are collected from 19 CAM journals in the Science Citation Index Expanded (SCI-E) database during 1980–2009, and 17,002 papers are identified for analysis. The study analyzes the document types, geographical and institutional distribution of the authorship, including international scientific collaboration. This study suggests that the major type of document is original article. The CAM papers are mostly published by North America, East Asia, and European countries, of which publications authored in East Asia are cited most. Country-wise, major contributors of CAM papers are from USA, People’s Republic of China, India, England and Germany. India has the highest CPP value, attracting high attentions in CAM community. This article also finds that international co-authorship in the CAM field has increased rapidly during this period. In addition, internationally collaborated publications generate higher citation impact than papers published by authors from single country. Finally, the research identifies productive institutions in CAM, and China Medical University located in Taiwan is the most productive organization.     

Usage metrics vs classical metrics: analysis of Russia’s research output

This paper discusses the results of a pilot project investigating Russian scholarly publications using the altmetric indicators “Usage Count Last 180 days” (U1) and “Usage Count Since 2013” (U2) introduced by Web of Science. We explored the relationship between citation impact and both types of usage counts. The data set consisted of 37,281 records (publications) indexed by SCI-E in 2015. Seven broad research areas were selected to observe citation patterns and usage counts. A significant difference was found between mean citations and mean usage counts (U2) in a few research areas. We discovered a significant Kendall rank correlation between the citation metrics and usage metrics at the article level. This correlation is particularly strong for the longer period usage metric (U2). We also analyzed the relationship between usage metrics and traditional journal-level citation metrics. Very weak correlation was observed.     

Citation time window choice for research impact evaluation

This paper aims to inform choice of citation time window for research evaluation, by answering three questions: (1) How accurate is it to use citation counts in short time windows to approximate total citations? (2) How does citation ageing vary by research fields, document types, publication months, and total citations? (3) Can field normalization improve the accuracy of using short citation time windows? We investigate the 31-year life time non-self-citation processes of all Thomson Reuters Web of Science journal papers published in 1980. The correlation between non-self-citation counts in each time window and total non-self-citations in all 31 years is calculated, and it is lower for more highly cited papers than less highly cited ones. There are significant differences in citation ageing between different research fields, document types, total citation counts, and publication months. However, the within group differences are more striking; many papers in the slowest ageing field may still age faster than many papers in the fastest ageing field. Furthermore, field normalization cannot improve the accuracy of using short citation time windows. Implications and recommendations for choosing adequate citation time windows are discussed.     

Using neural-network based paragraph embeddings for the calculation of within and between document similarities

Science mapping using document networks comes often with the implicit assumption that scientific papers are indivisible units with unique links to neighbour documents. Research on proximity in co-citation analysis and the study of lexical properties of sections and citation contexts indicate that this assumption doesn’t always hold. Moreover, the meaning of words and co-words depends on the context in which they appear. This study proposes the use of a neural network architecture for word and paragraph embeddings (Doc2Vec) for the measurement of similarity among those smaller units of analysis. It is shown that paragraphs in the “Introduction” and the “Discussion” Section are more similar to the abstract, that the similarity among paragraphs is related to -but not linearly- the distance between the paragraphs. The “Methodology” Section is least similar to the other sections. Abstracts of citing-cited documents are more similar than random pairs and the context in which a reference appears is most similar to the abstract of the cited document. This novel approach with higher granularity can be used for bibliometric aided retrieval and to assist in measuring interdisciplinarity through the application of network-based centrality measures.

     

Renewable energy research 1995–2009: a case study of wind power research in EU, Spain, Germany and Denmark

The paper reports the developments and citation patterns over three time periods of research on Renewable Energy generation and Wind Power 1995–2011 in EU, Spain, Germany and Denmark. Analyses are based on Web of Science and incorporate journal articles as well as conference proceeding papers. Scientometric indicators include publication collaboration ratios, top-player distribution as well as citedness and correspondence analyses of citing publications, relative citation impact, distributions of top-cited as well as top-citing institutions and publication sources and cluster analysis of citing title terms to map knowledge export areas. Findings show an increase in citation impact for Renewable Energy and Wind Power research albeit hampered by scarcely cited conference papers. Although EU maintains its global top position in producing Renewable Energy and Wind Power research the developments of EU and German world shares as well as citation impact are negative during the most recent 7 year period. During the same time the citation impact of Spain and Denmark increase and place both nations among the top-ranking countries in Wind Power research. Spain is the only EU country that increases its world production share from 2000. China is currently ranked three after EU and USA in research output, however with a very low citation impact. Spain, Denmark and Germany each demonstrates distinct collaboration patterns and publication source and citation distribution profiles. More than half the citations to EU Wind Power research are EU-self citations. An expected intensified EU collaboration in the Wind Energy field does not come about. The most productive research institutions in Denmark and Spain are also the most cited ones.