When the search for truth fails: A computer simulation of the impact of the publication bias on the meta-analysis of scientific literature

The production of scientific knowledge is a complex social process, where many actors contribute by their publications to the disclosure of the hidden truth. However, due to different methods, analysed samples, and control variables, empirical findings from this process are often contradictory. Thus, quantitative sciences use meta-analyses in order to extract the likely truth from a corpus of publications about a given research question. Unfortunately, this procedure is often impaired by different forms of the so-called publication bias: papers with null-results are sometimes not published due to the publication policy of journal editors and their boards. Similarly, articles with a high news value may have a better chance of being published, even if their findings finally prove to be wrong. Thus the publications used for meta-analyses are often distorted and lead to wrong conclusions about the truth. For this reason the present article develops a formal model of the effects of the publication bias on the results of meta-analyses. It is successfully tested with empirical data and used for studying the conditions, under which meta-analyses disclose, obscure, or revert the underlying truth. As a main result of the related computer simulations it turns out that the publication bias has for true zero-relations other consequences than for true non-zero relations. Moreover, there are situations where certain forms of the publication bias have unexpectedly favourable effects on the disclosure of the truth by meta-analyses.

     

Graph-based algorithms for ranking researchers: not all swans are white!

Scientific importance ranking has long been an important research topic in scientometrics. Many indices based on citation counts have been proposed. In recent years, several graph-based ranking algorithms have been studied and claimed to be reasonable and effective. However, most current researches fall short of a concrete view of what these graph-based ranking algorithms bring to bibliometric analysis. In this paper, we make a comparative study of state-of-the-art graph-based algorithms using the APS (American Physical Society) dataset. We focus on ranking researchers. Some interesting findings are made. Firstly, simple citation-based indices like citation count can return surprisingly better results than many cutting-edge graph-based ranking algorithms. Secondly, how we define researcher importance may have tremendous impacts on ranking performance. Thirdly, some ranking methods which at the first glance are totally different have high rank correlations. Finally, the data of which time period are chosen for ranking greatly influence ranking performance but still remains open for further study. We also try to give explanations to a large part of the above findings. The results of this study open a third eye on the current research status of bibliometric analysis.     

Algorithmic identification of Ph.D. thesis-related publications: a proof-of-concept study

In this study we propose and evaluate a method to automatically identify the journal publications that are related to a Ph.D. thesis using bibliographical data of both items. We build a manually curated ground truth dataset from German cumulative doctoral theses that explicitly list the included publications, which we match with records in the Scopus database. We then test supervised classification methods on the task of identifying the correct associated publications among high numbers of potential candidates using features of the thesis and publication records. The results indicate that this approach results in good match quality in general and with the best results attained by the “random forest” classification algorithm.

     

Phase unwrapping in three dimensions with application to InSAR time series

The problem of phase unwrapping in two dimensions has been studied extensively in the past two decades, but the three-dimensional (3D) problem has so far received relatively little attention. We develop here a theoretical framework for 3D phase unwrapping and also describe two algorithms for implementation, both of which can be applied to synthetic aperture radar interferometry (InSAR) time series. We test the algorithms on simulated data and find both give more accurate results than a two-dimensional algorithm. When applied to actual InSAR time series, we find good agreement both between the algorithms and with ground truth.     

Indirect decoding edges for one-shot shape acquisition

Determining the correspondence in coded structured light is challenging, but it is very important in one-shot techniques for 3D shape reconstruction. The problem of decoding stripe patterns can be modeled as matching two code sequences. We propose decoding edges indirectly based on the property of the stripe pattern, which can be represented as edge code, color code, or mixed code sequences. While traditional methods match two edge code sequences, indirect decoding matches two color sequences or mixed code sequences. The advantages of the proposed method, including a higher Hamming distance, enforced local coherence, and more code information, make indirect decoding excellent in performance. Previously, the lack of ground truth has prevented direct comparisons of different decoding algorithms. Here we obtain six benchmark datasets using the robust space-time analysis. Five decoding methods are quantitatively evaluated using the ground truth. The comparison results show that our method is robust for complex code situations and that it outperforms the state-of-the-art technique in this area.     

Ranking scientific publications considering the aging characteristics of citations

Ranking the significance of scientific publications has been a challenging topic for a long time. So far, many ranking methods have been proposed, one of which is the well-known PageRank algorithm. In this paper, we introduce aging characteristics to the PageRank algorithm via considering only the first 10 year citations when aggregating resource from different nodes. The validation of our new method is performed on the data of American Physical Society journals. The results indicate that taking into account aging characteristics improves the performance of the PageRank algorithm in terms of ranking accuracy for both papers and authors. Though our method is only applied to citation networks in this paper, it can be naturally used in many other real systems and similar improvements are expected.

     

The shape of the utility or value function in stochastic multicriteria acceptability analysis

Stochastic multicriteria acceptability analysis (SMAA) is a decision support method that allows representing uncertain, imprecise, and partially missing criteria measurements and preference information as probability distributions. In this paper, we test how the assumed shape of the utility or value function affects the results of SMAA in two different problem settings: identifying the most preferred alternative and ranking all the alternatives. A linear value function has been most frequently applied, because more precise shape information can be difficult to obtain in real-life applications. In this paper, we analyse one past real-life problem and a large number of randomly generated test problems of different size using additive functions of different shape. The shape varies from linear to increasingly concave and convex exponential utility or value functions corresponding to different attitudes on marginal value or risk. The results indicate that in most cases slight non-linearity does not significantly affect the results. The proposed method can be used for evaluating how robust a particular real-life decision problem is with respect to the shape of the function. Based on this information, it is possible to determine how accurately the DMs’ preferences need to be assessed in a particular problem, and if it is possible to assume a simple linear shape.     

Transmuting performance on manufacturing dimensions into business performance: An exploratory analysis of efficiency using DEA

This research examines efficiency in the transformation of performance on manufacturing competitive dimensions into business performance in the automotive supplier industry. Specifically, the study explores efficiency produced from data envelopment analysis (DEA) using cost, quality, time, flexibility and innovativeness performance as inputs and return on investment (ROI) and return on assets (ROA) as outputs. First tier suppliers to the 'Big 3' in North America were sampled. The 51 suppliers were categorized into three groups (high, medium and low) based on their efficiency. No significant group differences in quality, time, flexibility and innovativeness performance were found, but a significant difference does exist for cost. The results were further investigated by testing between-group differences for various cost reduction programmes, thereby identifying the levers and benchmarks for improving low efficiency firms.     

Efficient algorithms for approximate time separation of events

Finding bounds on time separation of events is a fundamental problem in the verification and analysis of asynchronous and concurrent systems. Unfortunately, even for systems without repeated events or choice, computing exact bounds on time separation of events is an intractable problem when both min and max type timing constraints are present. In this paper, we describe a method for approximating min and max type constraints, and develop a polynomial-time algorithm for computing approximate time separation bounds in choice-free systems without repeated events. Next, we develop a pseudo-polynomial time technique for analysing a class of asynchronous systems in which events repeat over time. Unlike earlier works, our algorithms can analyse systems with both min and max type timing constraints efficiently. Although the computed bounds are conservative in the worst-case, experimental results indicate that they are fairly accurate in practice. We present formal proofs of correctness of our algorithms, and demonstrate their efficiency and accuracy by applying them to a suite of benchmarks. A complete asynchronous chip has been modelled and analysed using the proposed technique, revealing potential timing problems (already known to designers) in the datapath design.     

In quest of new document relations: evaluating co-opinion relations between co-citations and its impact on Information retrieval effectiveness

Document relational network has been effective in retrieving and evaluating papers. Despite their effectiveness, relational measures, including co-citation, are far from ideal and need improvements. The assumption underlying the co-citation relation is the content relevance and opinion relatedness of cited and citing papers. This may imply existence of some kind of co-opinionatedness between co-cited papers which may be effective in improving the measure. Therefore, the present study tries to test the existence of this phenomenon and its role in improving information retrieval. To do so, based on CITREC, a medical test collection was developed consisting of 30 queries (seed documents) and 4823 of their co-cited papers. Using NLP techniques, the co-citances of the queries and their co-cited papers were analyzed and their similarities were computed by 4 g similarity measure. Opinion scores were extracted from co-citances using SentiWordnet. Also, nDCG values were calculated and then compared in terms of the citation proximity index (CPI) and co-citedness measures before and after being normalized by the co-opinionatedness measure. The reliability of the test collection was measured by generalizability theory. The findings suggested that a majority of the co-citations exhibited a high level of co-opinionatedness in that they were mostly similar either in their opinion strengths or in their polarities. Although anti-polar co-citations were not trivial in their number, a significantly higher number of the co-citations were co-polar, with a majority being positive. The evaluation of the normalization of the CPI and co-citedness by the co-opinionatedness indicated a generally significant improvement in retrieval effectiveness. While anti-polar similarity reduced the effectiveness of the measure, the co-polar similarity proved to be effective in improving the co-citedness. Consequently, the co-opinionatedness can be presented as a new document relation and used as a normalization factor to improve retrieval performance and research evaluation.

     

Pharmacology and Pharmacy: research and innovation analysis

Pharmacology and Pharmacy has a highly vertiginous evolution at international level. Its results are widely applied in the pharmaceutical industry, which is in the first places internationally. The study is aimed to analyse the behaviour of the leading institutions at global level in terms of production of papers and patents; as well as the impact that these contributions generate. The databases used were Science Citation Index and Derwent Innovation Index, both belonging to Web of Science platform. The InCite tool was used to analyse the period from 2000 to 2019. It was demonstrated that there is no correspondence among the highlighted institutions in patent applications, patents granted and number of citations, as well as among entities with the highest number of papers, highly cited papers and number of citations. Leadership in the discipline of Pharmacology and Pharmacy is neither determined by their highly productive institutions nor the impact generated by them.

     

A Novel Sensing and Data Fusion System for 3-D Arm Motion Tracking in Telerehabilitation

In this paper, we present a novel sensing and data fusion system to track 3-D arm motion in a telerehabilitation program. A particle filter (PF) algorithm is adopted in the system to fuse data from inertial and visual sensors in a probabilistic manner. It is able to propagate multimodal distributions of system states based on an ldquoimportance samplingrdquo technique by using sets of weighted particles. To avoid the problem of conventional PF algorithms that suffer from particle degeneracy and perform poorly in a narrow distribution situation, we adopt two strategies in our system, namely state space pruning and an arm physical geometry constraint. Experimental results show that the proposed PF framework outperforms other fusion methods and provides accurate results in comparison to the ground truth.     

A comparison of two bibliometric methods for mapping of the research front

Summary This paper builds on previous research concerned with the classification and specialty mapping of research fields. Two methods are put to test in order to decide if significant differences as to mapping results of the research front of a science field occur when compared. The first method was based on document co-citation analysis where papers citing co-citation clusters were assumed to reflect the research front. The second method was bibliographic coupling where likewise citing papers were assumed to reflect the research front. The application of these methods resulted in two different types of aggregations of papers: (1) groups of papers citing clusters of co-cited works and (2) clusters of bibliographically coupled papers. The comparision of the two methods as to mapping results was pursued by matching word profiles of groups of papers citing a particular co-citation cluster with word profiles of clusters of bibliographically coupled papers. Findings suggested that the research front was portrayed in two considerably different ways by the methods applied. It was concluded that the results in this study would support a further comparative study of these methods on a more detailed and qualitative ground. The original data set encompassed 73,379 articles from the fifty most cited environmental science journals listed in Journal Citation Report, science edition downloaded from the Science Citation Index on CD-ROM.     

Bi-objective partial flexible job shop scheduling problem: NSGA-II,NRGA, MOGA and PAES approaches

This paper deals with a problem of partial flexible job shop with the objective of minimising makespan and minimising total operation costs. This problem is a kind of flexible job shop problem that is known to be NP-hard. Hence four multi-objective, Pareto-based, meta-heuristic optimisation methods, namely non-dominated sorting genetic algorithm (NSGA-II), non-dominated ranked genetic algorithm (NRGA), multi-objective genetic algorithm (MOGA) and Pareto archive evolutionary strategy (PAES) are proposed to solve the problem with the aim of finding approximations of optimal Pareto front. A new solution representation is introduced with the aim of solving the addressed problem. For the purpose of performance evaluation of our proposed algorithms, we generate some instances and use some benchmarks which have been applied in the literature. Also a comprehensive computational and statistical analysis is conducted in order to analyse the performance of the applied algorithms in five metrics including non-dominated solution, diversification, mean ideal distance, quality metric and data envelopment analysis are presented. Data envelopment analysis is a well-known method for efficiently evaluating the effectiveness of multi-criteria decision making. In this study we proposed this method of assessment of the non-dominated solutions. The results indicate that in general NRGA and PAES have had a better performance in comparison with the other two algorithms.     

Predicting accident frequency at their severity levels and its application in site ranking using a two-stage mixed multivariate model

Accident prediction models (APMs) have been extensively used in site ranking with the objective of identifying accident hotspots. Previously this has been achieved by using a univariate count data or a multivariate count data model (e.g. multivariate Poisson-lognormal) for modelling the number of accidents at different severity levels simultaneously. This paper proposes an alternative method to estimate accident frequency at different severity levels, namely the two-stage mixed multivariate model which combines both accident frequency and severity models. The accident, traffic and road characteristics data from the M25 motorway and surrounding major roads in England have been collected to demonstrate the use of the two-stage model. A Bayesian spatial model and a mixed logit model have been employed at each stage for accident frequency and severity analysis respectively, and the results combined to produce estimation of the number of accidents at different severity levels. Based on the results from the two-stage model, the accident hotspots on the M25 and surround have been identified. The ranking result using the two-stage model has also been compared with other ranking methods, such as the naïve ranking method, multivariate Poisson-lognormal and fixed proportion method. Compared to the traditional frequency based analysis, the two-stage model has the advantage in that it utilises more detailed individual accident level data and is able to predict low frequency accidents (such as fatal accidents). Therefore, the two-stage mixed multivariate model is a promising tool in predicting accident frequency according to their severity levels and site ranking.     

Predicting accident frequency at their severity levels and its application in site ranking using a two-stage mixed multivariate model

Accident prediction models (APMs) have been extensively used in site ranking with the objective of identifying accident hotspots. Previously this has been achieved by using a univariate count data or a multivariate count data model (e.g. multivariate Poisson-lognormal) for modelling the number of accidents at different severity levels simultaneously. This paper proposes an alternative method to estimate accident frequency at different severity levels, namely the two-stage mixed multivariate model which combines both accident frequency and severity models. The accident, traffic and road characteristics data from the M25 motorway and surrounding major roads in England have been collected to demonstrate the use of the two-stage model. A Bayesian spatial model and a mixed logit model have been employed at each stage for accident frequency and severity analysis respectively, and the results combined to produce estimation of the number of accidents at different severity levels. Based on the results from the two-stage model, the accident hotspots on the M25 and surround have been identified. The ranking result using the two-stage model has also been compared with other ranking methods, such as the naïve ranking method, multivariate Poisson-lognormal and fixed proportion method. Compared to the traditional frequency based analysis, the two-stage model has the advantage in that it utilises more detailed individual accident level data and is able to predict low frequency accidents (such as fatal accidents). Therefore, the two-stage mixed multivariate model is a promising tool in predicting accident frequency according to their severity levels and site ranking.     

Update-based evolution control: A new fitness approximation method for evolutionary algorithms

Evolutionary algorithms are robust optimization methods that have been used in many engineering applications. However, real-world fitness evaluations can be computationally expensive, so it may be necessary to estimate the fitness with an approximate model. This article reviews design and analysis of computer experiments (DACE) as an approximation method that combines a global polynomial with a local Gaussian model to estimate continuous fitness functions. The article incorporates DACE in various evolutionary algorithms, to test unconstrained and constrained benchmarks, both with and without fitness function evaluation noise. The article also introduces a new evolution control strategy called update-based control that estimates the fitness of certain individuals of each generation based on the exact fitness values of other individuals during that same generation. The results show that update-based evolution control outperforms other strategies on noise-free, noisy, constrained and unconstrained benchmarks. The results also show that update-based evolution control can compensate for fitness evaluation noise.     

Bibliometric analysis of the Czech research publications from 1994 to 2005

We have compared bibliometric data of Czech research papers generated from 1994 to 2005 with papers from six other EU countries: Austria, Hungary, Poland, Finland, Ireland and Greece. The Czech Republic ranked the fifth in number of papers per thousand inhabitants and the sixth in number of citations/paper. Relatively the most cited were Czech papers from fields Engineering and Mathematics ranking the third, and Computer Science, Environment/Ecology and Molecular Biology ranking the fourth among 7 EU countries. Our analysis indicates that Czech research is lagging behind the leading EU countries, but its output is proportional to the R&D expenses.     

Choosing the right collaboration partner for innovation: a framework based on topic analysis and link prediction

Selecting the right collaboration partner is one of the most important contributors to success in collaborative innovation. Accordingly, numerous methods for selecting an appropriate partner have been developed to guide would-be collaborators in their search. Most rely on bibliographic information, which may be easier for that data is readily available and relatively normalized. However, with the benefit of today’s text mining and fusion techniques, it is possible to mine the content of papers and patents so as to result in far more nuanced and advantageous choices. In this article, we explore how to select partners for collaborative innovation by combining the characteristics of the authors of paper and patent documents as well as their content. Drawing on existing research, we developed a systematic framework that relies on topic analysis and link prediction. With a corpus of papers and patents assembled, the framework extracts correlated scientific and technological topics followed by a list of author institutions and a list of patentees. These organisations are parsed and evaluated using two indicators of innovation—capability and openness—to result in two separate ranked lists. Two integrated collaboration networks that include both author institutions and patentees are then built, and a link prediction method identifies missing links with a high likelihood of fruitful cooperation. A case study on hepatitis C virus research shows that the ranking procedure and the link prediction method can be used either together or separately to effectively identify collaborative innovation partners. Our results provide significant quantitative evidence for policymakers who are looking to foster cooperation between research institutions and/or high-tech enterprises. Our research may also serve as the basis for further in-depth research on collaborative innovation, R&D cooperation, and link prediction theories and methods.

     

Texture augmented analysis of high resolution satellite imagery in detecting invasive plant species

Abstract

During recent decades, a considerable number of alien species have been brought into Taiwan and have caused significant impacts to local ecosystems and biodiversity. High resolution satellite imagery can provide detailed spatial characteristics over a large area and has a great potential for accurate vegetation mapping. However, most traditional multispectral image classification techniques focus on spectral discrimination of ground objects and may overlook useful spatial information provided by high resolution images. To achieve the best result, analysis of high resolution imagery should also incorporate spatial variations of the data. Therefore, this paper has looked into using a texture augmented procedure to analyze a high resolution satellite (QuickBird) image in order to detect an invasive plant species (Leucaena leucocephala) in southern Taiwan. Samples of primary vegetation covers were selected from the image to determine suitable texture analysis parameters for extracting texture features helpful for classification. Validation with ground truth data showed that the analysis produced high accuracies in detecting the target plant species and overall classification for primary vegetation types within the study site.