Citations,research topics and active countries in software engineering: A bibliometrics study

Context: An enormous number of papers (more than 70,000) have been published in the area of Software Engineering (SE) since its inception in 1968. To better characterize and understand this massive research literature, there is a need for comprehensive bibliometrics assessments in this vibrant field.Objective: The objective of this study is to utilize automated citation and topic analysis to characterize the software engineering research literature over the years. While a few bibliometrics studies have appeared in the field of SE, this article aims to be the most comprehensive bibliometrics assessments in this vibrant field.Method: To achieve the above objective, we report in this paper a bibliometrics study with data collected from Scopus database consisting of over 70,000 articles. For thematic analysis, we used topic modeling to automatically generate the most probable topic distributions given the data.Results: We found that number of papers published per year has grown tremendously and currently 6000–7000 papers are published every year. At the same time, nearly half of the papers are not cited at all. Using text mining of articles titles, we found that currently the hot research topics in software engineering are: (1) web services, (2) mobile and cloud computing, (3) industrial (case) studies, (4) source code and (5) test generation. Finally, we found that a small share of large countries produce the majority of the papers in SE while small European countries are proportionally the most active in the area of SE, based on the number of papers.Conclusion: Due to large volumes of research in SE, we suggest using the automated analysis of bibliometrics as we have done in this paper. By picking out the most cited papers, we can present the land marks of SE and, with thematic analysis, we can characterize the entire field. This can be useful for students and other new comers to SE and for presenting our achievements to other disciplines. In particular, we see and report the value of such an analysis in situations where performing a full scale SLR is not feasible due to restrictions on time or to lack of exact research questions.     

Citation analysis and bibliometric approach for ant colony optimization from 1996 to 2010

To build awareness of the development of ant colony optimization (ACO), this study clarifies the citation and bibliometric analysis of research publications of ACO during 1996-2010. This study analysed 12,960 citations from a total of 1372 articles dealing with ACO published in 517 journals based on the databases of SCIE, SSCI and AH&CI, retrieved via the Web of Science. Bradford Law and Lotka’s Law, respectively, examined the distribution of journal articles and author productivity. Furthermore, this study determines the citation impact of ACO using parameters such as extent of citation received in terms of number of citations per study, distribution of citations over time, distribution of citations among domains, citation of authors, citation of institutions, highly cited papers and citing journals and impact factor of 12,960 citations. This study can help researchers to better understand the history, current status and trends of ACO in the advanced study of it.     

What’s up with software metrics? - A preliminary mapping study

Background

Many papers are published on the topic of software metrics but it is difficult to assess the current status of metrics research.

Aim

This paper aims to identify trends in influential software metrics papers and assess the possibility of using secondary studies to integrate research results.

Method

Search facilities in the SCOPUS tool were used to identify the most cited papers in the years 2000-2005 inclusive. Less cited papers were also selected from 2005. The selected papers were classified according factors such as to main topic, goal and type (empirical or theoretical or mixed). Papers classified as “Evaluation studies” were assessed to investigate the extent to which results could be synthesized.

Results

Compared with less cited papers, the most cited papers were more frequently journal papers, and empirical validation or data analysis studies. However, there were problems with some empirical validation studies. For example, they sometimes attempted to evaluate theoretically invalid metrics and fail to appreciate the importance of the context in which data are collected.

Conclusions

This paper, together with other similar papers, confirms that there is a large body of research related to software metrics. However, software metrics researchers may need to refine their empirical methodology before they can answer useful empirical questions.     

Summarization of scientific documents by detecting common facts in citations

Reading scientific articles is more time-consuming than reading news because readers need to search and read many citations. This paper proposes a citation guided method for summarizing multiple scientific papers. A phenomenon we can observe is that citation sentences in one paragraph or section usually talk about a common fact, which is usually represented as a set of noun phrases co-occurring in citation texts and it is usually discussed from different aspects. We design a multi-document summarization system based on common fact detection. One challenge is that citations may not use the same terms to refer to a common fact. We thus use term association discovering algorithm to expand terms based on a large set of scientific article abstracts. Then, citations can be clustered based on common facts. The common fact is used as a salient term set to get relevant sentences from the corresponding cited articles to form a summary. Experiments show that our method outperforms three baseline methods by ROUGE metric.     

Thirty Years of the International Journal of Intelligent Systems: A Bibliometric Review

The International Journal of Intelligent Systems was created in 1986. Today, the journal is 30 years old. To celebrate this anniversary, this study develops a bibliometric review of all of the papers published in the journal between 1986 and 2015. The results are largely based on the Web of Science Core Collection, which classifies leading bibliographic material by using several indicators including total number of publications and citations, the h‐index, cites per paper, and citing articles. The work also uses the VOS viewer software for visualizing the main results through bibliographic coupling and co‐citation. The results show a general overview of leading trends that have influenced the journal in terms of highly cited papers, authors, journals, universities and countries.     

An analysis of the most cited articles in software engineering journals - 2000

Citations and related work are crucial in any research to position the work and to build on the work of others. A high citation count is an indication of the influence of specific articles. The importance of citations means that it is interesting to analyze which articles are cited the most. Such an analysis has been conducted using the ISI Web of Science to identify the most cited software engineering journal articles published in 2000. The objective of the analysis is to identify and list the articles that have influenced others the most as measured by citation count. An understanding of which research is viewed by the research community as most valuable to build upon may provide valuable insights into what research to focus on now and in the future. Based on the analysis, a list of the 20 most cited articles is presented here. The intention of the analysis is twofold. First, to identify the most cited articles, and second, to invite the authors of the most cited articles in 2000 to contribute to a special issue of Information and Software Technology. Five authors have accepted the invitation and their articles appear in this special issue. Moreover, an analysis of the most cited software engineering journal articles in the last 20 years is presented. The presentation includes both the most cited articles in absolute numbers and the most cited articles when looking at the average number of citations per year. The article describing the SPIN model checker by G.J. Holzmann published in 1997 is first on both these lists.     

Keyword-based patent citation prediction via information theory

ABSTRACT

Patent citation shows how a technology impacts other inventions, so the number of patent citations (backward citations) is used in many technology prediction studies. Current prediction methods use patent citations, but since it may take a long time till a patent is cited by other inventors, identifying impactful patents based on their citations is not an effective way. The prediction method offered in this article predicts patent citations based on the content of patents. In this research, Reconstructability Analysis (RA), which is based on information theory and graph theory, is applied to predict patent citations based on keywords extracted from the abstracts of selected patents. After applying three classes of RA (variable-based analysis without and with loops and state-based analysis), nine specific IV states of a predicting model are extracted. These states involve the four keywords of “chamber”, “hous”, “main”, and “return”. Lastly, the abstracts of the patents are examined to identify the technical subjects relevant to smart building technologies for which these keywords are proxies.     

On the World Wide Web

ABSTRACT

An analysis of Web citations was performed to examine three aspects of scholarly and research communication: use of authoritative information, use of correct citation style, and availability of the cited sources for the purpose of verification. Results of the study indicate that a database or a repository of Web citations needs to be established.     

Bibliometric analysis of Human Factors (1970-2000): a quantitative description of scientific impact

Bibliometric analyses use the citation history of scientific articles as data to measure scientific impact. This paper describes a bibliometric analysis of the 1682 papers and 2413 authors published in Human Factors from 1970 to 2000. The results show that Human Factors has substantial relative scientific influence, as measured by impact, immediacy, and half-life, exceeding the influence of comparable journals. Like other scientific disciplines, human factors research is a highly stratified activity. Most authors have published only one paper, and many papers are cited infrequently, if ever. A small number of authors account for a disproportionately large number of the papers published and citations received. However, the degree of stratification is not as extreme as in many other disciplines, possibly reflecting the diversity of the human factors discipline. A consistent trend of more authors per paper parallels a similar trend in other fields and may reflect the increasingly interdisciplinary nature of human factors research and a trend toward addressing human-technology interaction in more complex systems. Ten of the most influential papers from each of the last 3 decades illustrate trends in human factors research. Actual or potential applications of this research include considerations for the publication and distribution policy of Human Factors.     

Software fault prediction metrics: A systematic literature review

ContextSoftware metrics may be used in fault prediction models to improve software quality by predicting fault location.ObjectiveThis paper aims to identify software metrics and to assess their applicability in software fault prediction. We investigated the influence of context on metrics’ selection and performance.MethodThis systematic literature review includes 106 papers published between 1991 and 2011. The selected papers are classified according to metrics and context properties.ResultsObject-oriented metrics (49%) were used nearly twice as often compared to traditional source code metrics (27%) or process metrics (24%). Chidamber and Kemerer’s (CK) object-oriented metrics were most frequently used. According to the selected studies there are significant differences between the metrics used in fault prediction performance. Object-oriented and process metrics have been reported to be more successful in finding faults compared to traditional size and complexity metrics. Process metrics seem to be better at predicting post-release faults compared to any static code metrics.ConclusionMore studies should be performed on large industrial software systems to find metrics more relevant for the industry and to answer the question as to which metrics should be used in a given context.     

Investigations about replication of empirical studies in software engineering: A systematic mapping study

ContextTwo recent mapping studies which were intended to verify the current state of replication of empirical studies in Software Engineering (SE) identified two sets of studies: empirical studies actually reporting replications (published between 1994 and 2012) and a second group of studies that are concerned with definitions, classifications, processes, guidelines, and other research topics or themes about replication work in empirical software engineering research (published between 1996 and 2012).ObjectiveIn this current article, our goal is to analyze and discuss the contents of the second set of studies about replications to increase our understanding of the current state of the work on replication in empirical software engineering research.MethodWe applied the systematic literature review method to build a systematic mapping study, in which the primary studies were collected by two previous mapping studies covering the period 1996–2012 complemented by manual and automatic search procedures that collected articles published in 2013.ResultsWe analyzed 37 papers reporting studies about replication published in the last 17 years. These papers explore different topics related to concepts and classifications, presented guidelines, and discuss theoretical issues that are relevant for our understanding of replication in our field. We also investigated how these 37 papers have been cited in the 135 replication papers published between 1994 and 2012.ConclusionsReplication in SE still lacks a set of standardized concepts and terminology, which has a negative impact on the replication work in our field. To improve this situation, it is important that the SE research community engage on an effort to create and evaluate taxonomy, frameworks, guidelines, and methodologies to fully support the development of replications.     

我国计算机科学发展态势文献计量分析

以Web of sciences、ESI（基本科学指标数据库）等数据库为统计分析源,对1996—2006年中国计算机科学的发展现状和发展态势从文献计量学的角度进行了统计分析,通过论文数量、被引频次、篇均被引频次和影响因子等项指标研究了计算机科学论文产出随时间的变化趋势、主要领域的发展状况及刊登论文期刊的影响力。     

The Most Cited IEEE Software Articles

This paper investigates the most cited papers in IEEE Software over its 25 year history. I find that the most cited paper has 135 citations. Further, using the H-Index to determine how many papers to analyze, I find an H-Index of 35 for IEEE Software, without omitting self citations, and an H-Index of 33 if self-citations are omitted. The percentage decrease between the total number of citations to the 35 and 33 papers respectively, was less than the percent decrease in the H-Index when self citations were omitted.     

Can we predict citation counts of environmental modelling papers? Fourteen bibliographic and categorical variables predict less than 30% of the variability in citation counts

We assessed 6122 environmental modelling papers published since 2005 to determine whether the number of citations each paper had received by September 2014 could be predicted with no knowledge of the paper's quality. A random forest was applied, using a range of easily quantified or classified variables as predictors. The 511 papers published in two key journals in 2008 were further analysed to consider additional variables. Papers with no differential equations received more citations. The topic of the paper, number of authors and publication venue were also significant. Ten other factors, some of which have been found significant in other studies, were also considered, but most added little to the predictive power of the models. Collectively, all factors predicted 16–29% of the variation in citation counts, with the remaining variance (the majority) presumably attributable to important subjective factors such as paper quality, clarity and timeliness.     

Potential and limitations of the ISBSG dataset in enhancing software engineering research: A mapping review

ContextThe International Software Benchmarking Standards Group (ISBSG) maintains a software development repository with over 6000 software projects. This dataset makes it possible to estimate a project’s size, effort, duration, and cost.ObjectiveThe aim of this study was to determine how and to what extent, ISBSG has been used by researchers from 2000, when the first papers were published, until June of 2012.MethodA systematic mapping review was used as the research method, which was applied to over 129 papers obtained after the filtering process.ResultsThe papers were published in 19 journals and 40 conferences. Thirty-five percent of the papers published between years 2000 and 2011 have received at least one citation in journals and only five papers have received six or more citations. Effort variable is the focus of 70.5% of the papers, 22.5% center their research in a variable different from effort and 7% do not consider any target variable. Additionally, in as many as 70.5% of papers, effort estimation is the research topic, followed by dataset properties (36.4%). The more frequent methods are Regression (61.2%), Machine Learning (35.7%), and Estimation by Analogy (22.5%). ISBSG is used as the only support in 55% of the papers while the remaining papers use complementary datasets. The ISBSG release 10 is used most frequently with 32 references. Finally, some benefits and drawbacks of the usage of ISBSG have been highlighted.ConclusionThis work presents a snapshot of the existing usage of ISBSG in software development research. ISBSG offers a wealth of information regarding practices from a wide range of organizations, applications, and development types, which constitutes its main potential. However, a data preparation process is required before any analysis. Lastly, the potential of ISBSG to develop new research is also outlined.     

Publication in the Online First Directory

Recent technological progress has enabled us to meet authors' requirements for shorter publication times. Thus we are pleased to announce that Springer-Verlag is introducing the Online First (OF) publication concept for Acta Informatica. This new facility has been installed by Springer's Internet service (http://www.springerlink.com). From now on papers will be published in electronic form as OF papers shortly after proofs have been corrected by the author and returned to Springer. This means a significant reduction in the publication time. At this stage, OF papers will have no citation line (volume, pages, year) or page numbers but will be marked by an international identification code which can used for citation purposes (see below). A line indicating the exact OF-publication date will be printed beneath the title.For publication in the subsequent print version, the citation line and page numbers will be added, but the content of the article will remain unchanged. Authors will be informed at the proofreading stage that after electronic publication they cannot change the content of a paper, nor can they withdraw it from publication.Each article will be announced by Springer's Alert service. Access to the full papers is restricted to subscribers of the printed version.An OF paper is a genuine publication and not a new form of preprint. It is copyright-protected and it can be cited exactly like any printed paper. An OF paper is citeable by a "Digital Object Identifier" (or DOI), a unique and persistent international identification code, included in both the print and electronic versions. The DOI can also be used to create hyperlinks to OF papers: although files may be moved to other locations on the Springer server after print publication, the DOI is never changed.September 2003     

Time to stop researching the important things?

From 2010 government funding for UK research will be based partly on metrics. One of the key metrics will be citations, with research income and postgraduate student numbers also assessed. While citation metrics seem sensible, there is a serious issue of what topics one researches, the methods used and the way in which a move to the use of citations will induce a change in research behaviour – especially a move to the dull middle ground. This paper investigates the potential impact of such a move on niche areas of IS research – here research on small- and medium-sized enterprises. However, the arguments hold for any new, emerging, or non-mainstream research area. The impacts of citation metrics on publication outlets and research method choice are assessed.     

Dynamics of citation distribution

We study the citation dynamics of scientific publications over the years. We propose a simple cellular automaton model featuring a combination of two distinct mechanisms, i.e. the random assignment and the preferential attachment, to investigate the dynamics of journal citation. Different from most previous studies focusing on highly cited papers, we analyze the time evolution of the entire citation distribution. Empirical data can be well reproduced by numerical simulations. Within the linear regime of the Cited Half-Life, a steady accumulation of citations can be expected. Moreover, within this linear regime, the ratio between the above two mechanisms is a constant. Besides the average citation represented by the Impact Factor, such a constant ratio can also be a characteristic of the journal.     

An analysis of the most cited articles in software engineering journals – 2002

Citations and related work are crucial in any research to position the work and to build on the work of others. A high citation count is an indication of the influence of specific articles. The importance of citations means that it is interesting to analyze which articles are cited the most. Such an analysis has been conducted using the ISI Web of Science to identify the most cited software engineering journal articles published in 2002. The objective of the analysis is to identify and list the articles that have influenced others the most as measured by citation count. An understanding of which research is viewed by the research community as most valuable to build upon may provide valuable insights into what research to focus on now and in the future. Based on the analysis, a list of the 20 most cited articles is presented here. The intention of the analysis is twofold. First, to identify the most cited articles, and second, to invite the authors of the most cited articles in 2002 to contribute to a special section of Information and Software Technology. Six authors have accepted the invitation and their articles appear in this special section.