Pruning Algorithms for Rule Learning

Pre-pruning and Post-pruning are two standard techniques for handling noise in decision tree learning. Pre-pruning deals with noise during learning, while post-pruning addresses this problem after an overfitting theory has been learned. We first review several adaptations of pre- and post-pruning techniques for separate-and-conquer rule learning algorithms and discuss some fundamental problems. The primary goal of this paper is to show how to solve these problems with two new algorithms that combine and integrate pre- and post-pruning.     

Configuration vs. adaptation for business process variant maintenance: An empirical study

Many approaches for process variant management employ a reference model for deriving a target variant either using configuration or adaptation mechanisms. What is missing at this stage is empirical insight into their relative strengths and weaknesses. Our paper addresses this gap. We selected C-YAWL and vBPMN for a comparative, empirical user study. Both approaches center on a reference process, but provide different types of configuration and adaptation mechanisms as well as modularization support. Along with this aspect, we investigate the effect of model complexity and professional level on human process variant modeling performance. Given unlimited processing time, we could not show that complexity or the participant's professional level significantly impacts the task success rate or user contentment. Yet, an effect of model complexity can be noted on the execution speed for typical variant maintenance tasks like the insertion and deletion of process steps. For each of the performance measures of success rate, user contentment and execution speed, vBPMN performs significantly better than C-YAWL. We argue that this is due to vBPMN's advanced modularization support in terms of pattern-based process adaptations to construct process variants. These insights are valuable for advancing existing modeling approaches and selecting between them.     

Corrective classification: Learning from data imperfections with aggressive and diverse classifier ensembling

Learning from imperfect (noisy) information sources is a challenging and reality issue for many data mining applications. Common practices include data quality enhancement by applying data preprocessing techniques or employing robust learning algorithms to avoid developing overly complicated structures that overfit the noise. The essential goal is to reduce noise impact and eventually enhance the learners built from noise-corrupted data. In this paper, we propose a novel corrective classification (C2) design, which incorporates data cleansing, error correction, Bootstrap sampling and classifier ensembling for effective learning from noisy data sources. C2 differs from existing classifier ensembling or robust learning algorithms in two aspects. On one hand, a set of diverse base learners of C2 constituting the ensemble are constructed via a Bootstrap sampling process; on the other hand, C2 further improves each base learner by unifying error detection, correction and data cleansing to reduce noise impact. Being corrective, the classifier ensemble is built from data preprocessed/corrected by the data cleansing and correcting modules. Experimental comparisons demonstrate that C2 is not only more accurate than the learner built from original noisy sources, but also more reliable than Bagging [4] or aggressive classifier ensemble (ACE) [56], which are two degenerated components/variants of C2. The comparisons also indicate that C2 is more stable than Boosting and DECORATE, which are two state-of-the-art ensembling methods. For real-world imperfect information sources (i.e. noisy training and/or test data), C2 is able to deliver more accurate and reliable prediction models than its other peers can offer.     

Adaptive networks for physical modeling

This paper presents an original link between neural networks theory and mechanical modeling networks. The problem is to find the parameters characterizing mechanical structures in order to reproduce given mechanical behaviors. Replacing “neural” units with mechanically based units and applying classical learning algorithms dedicated to supervised dynamic networks to these mechanical networks allows us to find the parameters for a physical model. Some new variants of real-time recurrent learning (RTRL) are also introduced, based on mechanical principles.

The notion of interaction during learning is discussed at length and the results of tests are presented. Instead of the classical {machine learning system, environment} pair, we propose to study the {machine learning system, human operator, environment} triplet.

Experiments have been carried out in simulated scenarios and some original experiments with a force-feedback device are also described.     

Inductive transfer for learning Bayesian networks

In several domains it is common to have data from different, but closely related problems. For instance, in manufacturing, many products follow the same industrial process but with different conditions; or in industrial diagnosis, where there is equipment with similar specifications. In these cases it is common to have plenty of data for some scenarios but very little for others. In order to learn accurate models for rare cases, it is desirable to use data and knowledge from similar cases; a technique known as transfer learning. In this paper we propose an inductive transfer learning method for Bayesian networks, that considers both structure and parameter learning. For structure learning we use conditional independence tests, by combining measures from the target task with those obtained from one or more auxiliary tasks, using a novel weighted sum of the conditional independence measures. For parameter learning, we propose two variants of the linear pool for probability aggregation, combining the probability estimates from the target task with those from the auxiliary tasks. To validate our approach, we used three Bayesian networks models that are commonly used for evaluating learning techniques, and generated variants of each model by changing the structure as well as the parameters. We then learned one of the variants with a small dataset and combined it with information from the other variants. The experimental results show a significant improvement in terms of structure and parameters when we transfer knowledge from similar tasks. We also evaluated the method with real-world data from a manufacturing process considering several products, obtaining an improvement in terms of log-likelihood between the data and the model when we do transfer learning from related products.     

Process diagnostics using trace alignment: Opportunities, issues, and challenges

Business processes leave trails in a variety of data sources (e.g., audit trails, databases, and transaction logs). Hence, every process instance can be described by a trace, i.e., a sequence of events. Process mining techniques are able to extract knowledge from such traces and provide a welcome extension to the repertoire of business process analysis techniques. Recently, process mining techniques have been adopted in various commercial BPM systems (e.g., BPM|one, Futura Reflect, ARIS PPM, Fujitsu Interstage, Businesscape, Iontas PDF, and QPR PA). Unfortunately, traditional process discovery algorithms have problems dealing with less structured processes. The resulting models are difficult to comprehend or even misleading. Therefore, we propose a new approach based on trace alignment. The goal is to align traces in such a way that event logs can be explored easily. Trace alignment can be used to explore the process in the early stages of analysis and to answer specific questions in later stages of analysis. Hence, it complements existing process mining techniques focusing on discovery and conformance checking. The proposed techniques have been implemented as plugins in the ProM framework. We report the results of trace alignment on one synthetic and two real-life event logs, and show that trace alignment has significant promise in process diagnostic efforts.     

Enhanced simulated-annealing-based algorithms and their applications to process planning in reconfigurable manufacturing systems

Capabilities of enhanced simulated-annealing-based algorithms in solving process planning problems in reconfigurable manufacturing are investigated. The algorithms are enhanced by combining variants of the simulated annealing technique with other algorithm concepts such as (i) knowledge exploitation and (ii) parallelism. Four configurations of simulated annealing algorithms are devised and engaged to solve an instance of a process planning problem in reconfigurable manufacturing systems. These configurations include; a basic simulated annealing algorithm, a variant of the basic simulated annealing algorithm, a variant of the simulated annealing algorithm coupled with auxiliary knowledge and a variant of the simulated annealing algorithm implemented in a quasi-parallel architecture. Although differences in performances were observed, the implemented algorithms are capable of obtaining good solutions in reasonable time. Experimental results show that the performances of the variants of simulated annealing based algorithms are better in comparison to a basic simulated annealing algorithm. A computational analysis and comparison using ANOVA indicates that improvements towards a better optimal solution can be gained by implementing variants of the simulated annealing algorithm. In addition, little speed gains can be obtained by implementing variants of the simulated annealing algorithms that are coupled with other algorithmic concepts.     

Using ensembles for problems with characterizable changes in data distribution: A case study on quantification

Ensemble methods are widely applied to supervised learning tasks. Based on a simple strategy they often achieve good performance, especially when the single models comprising the ensemble are diverse. Diversity can be introduced into the ensemble by creating different training samples for each model. In that case, each model is trained with a data distribution that may be different from the original training set distribution. Following that idea, this paper analyzes the hypothesis that ensembles can be especially appropriate in problems that: (i) suffer from distribution changes, (ii) it is possible to characterize those changes beforehand. The idea consists in generating different training samples based on the expected distribution changes, and to train one model with each of them. As a case study, we shall focus on binary quantification problems, introducing ensembles versions for two well-known quantification algorithms. Experimental results show that these ensemble adaptations outperform the original counterpart algorithms, even when trivial aggregation rules are used.     

A learning from demonstration framework for adaptive task and motion planning in varying package-to-order scenarios

Current advances in Task and Motion Planning (TAMP) framework often rely on a specific and static task structure. A task structure is a sequence of how work pieces should be manipulated towards achieving a goal. Such systems can be problematic when task structures change as a result of human performance during human-robot collaboration scenarios in manufacturing or when redundant objects are present in the workspace, for example, during a Package-To- Order scenario with the same object type fulfilling different package configurations. In this paper, we propose a novel integrated TAMP framework that supports learning from human demonstrations while tackling variations in object positions and product configurations during massive-Package-To-Order (mPTO) scenarios in manufacturing as well as during human-robot collaboration scenarios. We design and apply a Graph Neural Network(GNN) based high-level reasoning module that is capable of handling variant goal configurations and can generalize to different task structures. Moreover, we also built a two-level motion module which can produce flexible and collision-free trajectories based on important features and task labels produced by the reasoning module. Through simulations and physical experiments, we show that our framework holds several advantages when compared with state-of-the-art previous work. The advantages include sample-efficiency and generalizability to unseen goal configurations as well as task structures.