Using localizing learning to improve supervised learning algorithms

Slow learning of neural-network function approximators can frequently be attributed to interference, which occurs when learning in one area of the input space causes unlearning in another area. To mitigate the effect of unlearning, this paper develops an algorithm that adjusts the weights of an arbitrary, nonlinearly parameterized network such that the potential for future interference during learning is reduced. This is accomplished by the reduction of a biobjective cost function that combines the approximation error and a term that measures interference. An analysis of the algorithm's convergence properties shows that learning with this algorithm reduces future unlearning. The algorithm can be used either during online learning or can be used to condition a network to have immunity from interference during a future learning stage. A simple example demonstrates how interference manifests itself in a network and how less interference can lead to more efficient learning. Simulations demonstrate how this new learning algorithm speeds up the training in various situations due to the extra cost function term.     

Accurate estimates of the generalization ability for symmetric set of predictors and randomized learning algorithms

The main issue of the combinatorial approach to overfitting is to obtain computationally efficient formulas for overfitting probabilities. A group-theoretical approach is proposed to simplify derivation of such formulas when the set of predictors has a certain group of symmetries. Examples of the sets are given. The general estimate of overfitting probability is proved for the randomized learning algorithm. It is applied to four model sets of predictors—a layer of the Boolean cube, the Boolean cube, the unimodal chain, and a bundle of monotonic chains.     

Temporal generalization of sub-pixel vegetation mapping with multiple machine learning and atmospheric correction algorithms

Temporal generalization allows a trained classification algorithm to be applied to multiple images across time to derive reliable classification map products. It is a challenging remote-sensing research topic since the results are dependent on the selection of atmospheric correction methods, classification algorithms, validation processes, and their varying combinations. This study examined the temporal generalization of sub-pixel vegetation mapping using multiple Landsat images (1990, 1996, 2004, and 2010). All Landsat images were processed with two atmospheric correction methods: simple dark object subtraction (DOS) and the Landsat Ecosystem Disturbance Adaptive Processing System (LEDAPS) algorithm. For the sub-pixel vegetation mapping of the 2004 Landsat image, we used high-resolution OrbView-3 images as a training/validation data set and compared three machine learning algorithms (neural networks, random forests, and classification and regression trees) for their classification performance. The trained classifiers were then applied to other Landsat images (1990, 1996, and 2010) to derive sub-pixel vegetation map products. For the 2004 Landsat image classification, cross-validation shows similar classification results for neural networks (root mean square error (RMSE) = 0.099) and random forests (RMSE = 0.100) algorithms, and both are better than classification and regression trees (RMSE = 0.123). Pseudo-invariant pixels between 2004 and 2010 were used as validation points to evaluate the temporal generalizability of classification algorithms. Simple DOS and LEDAPS atmospheric correction resulted in similar accuracy statistics. The neural-network-based classifier performed best in generating reliable sub-pixel vegetation map products across time.     

Using AUC and accuracy in evaluating learning algorithms

The area under the ROC (receiver operating characteristics) curve, or simply AUC, has been traditionally used in medical diagnosis since the 1970s. It has recently been proposed as an alternative single-number measure for evaluating the predictive ability of learning algorithms. However, no formal arguments were given as to why AUC should be preferred over accuracy. We establish formal criteria for comparing two different measures for learning algorithms and we show theoretically and empirically that AUC is a better measure (defined precisely) than accuracy. We then reevaluate well-established claims in machine learning based on accuracy using AUC and obtain interesting and surprising new results. For example, it has been well-established and accepted that Naive Bayes and decision trees are very similar in predictive accuracy. We show, however, that Naive Bayes is significantly better than decision trees in AUC. The conclusions drawn in this paper may make a significant impact on machine learning and data mining applications.     

Using Coplink to analyze criminal-justice data

The paper discusses the Coplink system. The system applies a concept space-a statistics-based, algorithmic technique that identifies relationships between suspects, victims, and other pertinent data-to accelerate criminal investigations and enhance law enforcement efforts. The Coplink concept space application, which began as a research project, has evolved into a real-time system being used in everyday police work. Coplink CS has been successfully deployed at the Tucson Police Department, where crime analysts, officers, detectives, and sergeants from 16 departmental units use the technology voluntarily as part of their daily investigative routine     

The ensemble approach to neural-network learning and generalization

A method is suggested for learning and generalization with a general one-hidden layer feedforward neural network. This scheme encompasses the use of a linear combination of heterogeneous nodes having randomly prescribed parameter values. The learning of the parameters is realized through adaptive stochastic optimization using a generalization data set. The learning of the linear coefficients in the linear combination of nodes is achieved with a linear regression method using data from the training set. One node is learned at a time. The method allows for choosing the proper number of net nodes, and is computationally efficient. The method was tested on mathematical examples and real problems from materials science and technology.     

Learning equilibrium as a generalization of learning to optimize

We argue that learning equilibrium is an appropriate generalization to multi-agent systems of the concept of learning to optimize in single-agent setting. We further define and discuss the concept of weak learning equilibrium.     

Using edit distance to analyze card sorts

Abstract: Card sorts are a knowledge elicitation technique in which participants are given a collection of items and are asked to partition them into groups based on their own criteria. Information about the participant's knowledge structure is inferred from the groups formed and the names used to describe the groups through various methods ranging from simple quantitative statistical measures (e.g. co‐occurrence frequencies) to complex qualitative methods (e.g. content analysis on the group names). This paper introduces a new technique for analyzing card sort data that uses quantitative measures to discover rich qualitative results. This method is based upon a distance metric between sorts that allows one to measure the similarity of groupings and then look for clusters of closely related sorts across individuals. By using software for computing these clusters, it is possible to identify common concepts across individuals, despite the use of different terminology.     

An introduction to kernel-based learning algorithms

This paper provides an introduction to support vector machines, kernel Fisher discriminant analysis, and kernel principal component analysis, as examples for successful kernel-based learning methods. We first give a short background about Vapnik-Chervonenkis theory and kernel feature spaces and then proceed to kernel based learning in supervised and unsupervised scenarios including practical and algorithmic considerations. We illustrate the usefulness of kernel algorithms by discussing applications such as optical character recognition and DNA analysis.     

Using distributed raspberry PIs to enable low-cost energy-efficient machine learning algorithms for scientific articles recommendation

In recent decades, machine learning has become a crucial factor in terms of automating business operations and assisting in the decision-making process within an organization. With the huge volume of data generated at an unprecedented rate has motivated researchers and industry analysts to constantly develop effective and efficient analytical models machine learning techniques. This study adds to the data mining community by evaluating some of the most significant text mining techniques and presenting a predictive model that will supposedly ease the process of literature review for researchers. In addition, it compares the execution of the model in terms of cost, energy consumption, accuracy and scalability in three different environments, namely, google cloud instance, google cloud functions and distributed raspberry PIs. Results yielded in our study showed that distributed Raspberry PIs can have a highly positive impact in terms of lowering costs and being energy efficient. On one hand, we found out that machine learning algorithms can be adapted and run on distributed raspberry PIs with low cost and low energy consumption compared to cloud alternatives. On the other hand, this solution does not offer great scalability and it requires more time on management, deployment and configuration. The distributed Raspberry PIs also showed bad performance on execution time compared to the other alternatives when comes to high processing power.     

Using an electronic spreadsheet to analyze manufacturing flow systems

Contemporary use of electronic spreadsheets has centered on financial- and accounting-type analysis. Models are described in this paper that facilitate the use of an electronic spreadsheet for analyzing the flow of work in process through manufacturing system. It is shown that the spreadsheet-modeled analytic approach works best for the flow shop configuration: however, some job shop deviations can be considered. Stochastic elements must be approximated, but results are surprisingly close to those that come from the same system modeled with FORTRAN-based languages such as SLAM or INSIGHT. [A. A. Pritsker & C. D. Pegden, Introduction to Simulation and SLAM. Halsted Press, New York (1979); S. D. Roberts, Simulation Modeling and Analysis with INSIGHT. Regenstrief Institute for Health Care, Indianapolis (1982).] The advantages, limitations and assumptions of spreadsheet simulation and analysis models are discussed at length.     

Using model-checkers to generate and analyze property relevant test-cases

Verification is applied to software as a proof method with respect to its requirements. Software testing is necessary because verification is often infeasible. Automation is desirable since the complexity and the effort involved are significant. However, automated software testing is commonly used to ensure confidence in the conformance of an implementation to an abstract model, not to its requirement properties. In this paper, we introduce the notion of property relevance of test-cases. Property relevant test-cases can be used to determine property violations. It is shown how to detect the properties relevant to a test-case. New coverage criteria based on property relevance are introduced. Automated generation of test-suites satisfying these criteria is also presented. Finally, feasibility is illustrated with an empirical evaluation.

适合近红外光谱数据特征的降维方法对比分析

因近红外光谱具有波长点多、谱带归属困难、光谱重叠严重及光谱分布结构未知等问题,在进行近红外光谱关键特征提取和数据特征空间映射时难以准确获知合适降维方法。为了解决该问题,本文对比分析了典型线性和非线性降维方法 ,并用烟叶近红外光谱数据从数据降维可视化和分类准确性识别率角度分别进行了实验验证。结果表明,线性降维算法,特别是PCA、LDA算法,比较适合应用于烟叶近红外光谱降维分析中,非线性降维算法因其泛化学习能力与推广能力差以及本征维数估计困难不适合应用于近红外光谱降维分析。     

Using the k-core decomposition to analyze the static structure of large-scale software systems

The structures of large-scale software systems are complex and difficult to understand for designers and developers. Traditional software metrics can hardly describe the structural complexity in modern software, and finding a quantitative method to depict and analyze large-scale software is still a challenge. In this paper we use complex networks theory to study software structure; in particular, we visualize the software graph and use the k-core to study it based on a recursive pruning of the least connected vertices. Different types of software are analyzed and some new noticeable properties are found, such as similar coreness, the vital highest core, high-core connecting tendency, and evolution stability. These features suggest that software is organized in a defined hierarchy of increasing centrality from outside to inside. The k-core analysis can help developers to improve software understanding, instruction test, and measurement and evaluation of the system in development.     

Using model checking to analyze the system behavior of the LHC production grid

DIRAC (Distributed Infrastructure with Remote Agent Control) is the grid solution designed to support production activities as well as user data analysis for the Large Hadron Collider “beauty” experiment. It consists of cooperating distributed services and a plethora of light-weight agents delivering the workload to the grid resources. Services accept requests from agents and running jobs, while agents actively fulfill specific goals. Services maintain database back-ends to store dynamic state information of entities such as jobs, queues, or requests for data transfer. Agents continuously check for changes in the service states and react to these accordingly. The logic of each agent is rather simple; the main source of complexity lies in their cooperation. These agents run concurrently and communicate using the services’ databases as a shared memory for synchronizing the state transitions. Despite the effort invested in making DIRAC reliable, entities occasionally get into inconsistent states. Tracing and fixing such behaviors is difficult, given the inherent parallelism among the distributed components and the size of the implementation.In this paper we present an analysis of DIRAC with mCRL2, process algebra with data. We have reverse engineered two critical and related DIRAC subsystems, and subsequently modeled their behavior with the mCRL2 toolset. This enabled us to easily locate race conditions and livelocks which were confirmed to occur in the real system. We further formalized and verified several behavioral properties of the two modeled subsystems.     

机器学习算法在药物专利分类中的应用研究

为实现药物专利的自动分类,本文结合药物专利的特点,研究了机器学习算法如何应用于药物专利分类。将2000余份药物专利按疗效分类,选取其中五类作为训练样本,对每一类提取特征文本,利用向量空间模型将非结构化的文本数字化,用支持向量机、Naive Bayes、RBFNetwork三种机器学习算法,分别测试专利样本的分类,使用5重交叉验证比较了三种算法的查准率(precision)和召回率(recall),结果表明支持向量机的分类效果最好。将机器学习算法应用于药物化学专利分类,有助于提高药物化学专利信息检索的效率。     

Convergence of learning algorithms with constant learning rates

The behavior of neural network learning algorithms with a small, constant learning rate, epsilon, in stationary, random input environments is investigated. It is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation, in the sense of weak convergence of random processes as epsilon tends to zero. As applications, backpropagation in feedforward architectures and some feature extraction algorithms are studied in more detail.