Learning from demonstration in robots: Experimental comparison of neural architectures

Robots have played an important role in the automation of computer aided manufacturing. The classical robot control implementation involves an expensive key step of model-based programming. An intuitive way to reduce this expensive exercise is to replace programming with machine learning of robot actions from demonstration where a (learner) robot learns an action by observing a demonstrator robot performing the same. To achieve this learning from demonstration (LFD) different machine learning techniques such as Artificial Neural Networks (ANN), Genetic Algorithms, Hidden Markov Models, Support Vector Machines, etc. can be used. This piece of work focuses exclusively on ANNs. Since ANNs have many standard architectural variations divided into two basic computational categories namely the recurrent networks and feed-forward networks, representative networks from each have been selected for study, i.e. Feed Forward Multilayer Perceptron (FF) network for feed-forward networks category and Elman (EL), and Nonlinear Autoregressive Exogenous Model (NARX) networks for the recurrent networks category. The main objective of this work is to identify the most suitable neural architecture for application of LFD in learning different robot actions. The sensor and actuator streams of demonstrated action are used as training data for ANN learning. Consequently, the learning capability is measured by comparing the error between demonstrator and corresponding learner streams. To achieve fairness in comparison three steps have been taken. First, Dynamic Time Warping is used to measure the error between demonstrator and learner streams, which gives resilience against translation in time. Second, comparison statistics are drawn between the best, instead of weight-equal, configurations of competing architectures so that learning capability of any architecture is not forced handicap. Third, each configuration's error is calculated as the average of ten trials of all possible learning sequences with random weight initialization so that the error value is independent of a particular sequence of learning or a particular set of initial weights. Six experiments are conducted to get a performance pattern of each architecture. In each experiment, a total of nine different robot actions were tested. Error statistics thus obtained have shown that NARX architecture is most suitable for this learning problem whereas Elman architecture has shown the worst suitability. Interestingly the computationally lesser MLP gives much lower and slightly higher error statistics compared to the computationally superior Elman and NARX neural architectures, respectively.     

The convergence analysis and specification of the Population-Based Incremental Learning algorithm

In this paper, we investigate the global convergence properties in probability of the Population-Based Incremental Learning (PBIL) algorithm when the initial configuration p⁽⁰⁾ is fixed and the learning rate α is close to zero. The convergence in probability of PBIL is confirmed by the experimental results. This paper presents a meaningful discussion on how to establish a unified convergence theory of PBIL that is not affected by the population and the selected individuals.     

Rolling element bearing fault diagnosis using wavelet transform

This paper is focused on fault diagnosis of ball bearings having localized defects (spalls) on the various bearing components using wavelet-based feature extraction. The statistical features required for the training and testing of artificial intelligence techniques are calculated by the implementation of a wavelet based methodology developed using Minimum Shannon Entropy Criterion. Seven different base wavelets are considered for the study and Complex Morlet wavelet is selected based on minimum Shannon Entropy Criterion to extract statistical features from wavelet coefficients of raw vibration signals. In the methodology, firstly a wavelet theory based feature extraction methodology is developed that demonstrates the information of fault from the raw signals and then the potential of various artificial intelligence techniques to predict the type of defect in bearings is investigated. Three artificial intelligence techniques are used for faults classifications, out of which two are supervised machine learning techniques i.e. support vector machine, learning vector quantization and other one is an unsupervised machine learning technique i.e. self-organizing maps. The fault classification results show that the support vector machine identified the fault categories of rolling element bearing more accurately and has a better diagnosis performance as compared to the learning vector quantization and self-organizing maps.     

Learning automata-based solutions to the optimal web polling problem modelled as a nonlinear fractional knapsack problem

We consider the problem of polling web pages as a strategy for monitoring the world wide web. The problem consists of repeatedly polling a selection of web pages so that changes that occur over time are detected. In particular, we consider the case where we are constrained to poll a maximum number of web pages per unit of time, and this constraint is typically dictated by the governing communication bandwidth, and by the speed limitations associated with the processing. Since only a fraction of the web pages can be polled within a given unit of time, the issue at stake is one of determining which web pages are to be polled, and we attempt to do it in a manner that maximizes the number of changes detected. We solve the problem by first modelling it as a stochastic nonlinear fractional knapsack problem. We then present an online learning automata (LA) system, namely, the hierarchy of twofold resource allocation automata (H-TRAA), whose primitive component is a twofold resource allocation automaton (TRAA). Both the TRAA and the H-TRAA have been proven to be asymptotically optimal. Finally, we demonstrate empirically that the H-TRAA provides orders of magnitude faster convergence compared to the learning automata knapsack game (LAKG) which represents the state-of-the-art for this problem. Further, in contrast to the LAKG, the H-TRAA scales sub-linearly. Based on these results, we believe that the H-TRAA has also tremendous potential to handle demanding real-world applications, particularly those which deal with the world wide web.     

Design and evaluation of a collaborative learning environment

Collaboration becomes an essential competency in the current knowledge society. In this study, a collaborative learning environment was designed to facilitate students in group collaboration. Instructional support strategies of friendship and meaningful learning tasks were applied to promote collaboration. Scaffolding strategies such as writing progress reports and developing product versions were used to coordinate and monitor the collaborative learning environment. In addition, the online file sharing tool DriveHQ was used to facilitate collaboration. Four classes of students used the learning environment to complete their final project in pairs. Results showed friendship and meaningful learning tasks helped to promote individual accountability and positive interdependence, and the use of progress reports and product versions was useful for coordinating and monitoring the learning process. However, additional strategies might be needed for more effective collaboration to take place. Issues involved in this study are discussed.     

Saving time or innovating practice: Investigating perceptions and uses of Learning Management Systems

Learning Management Systems (LMS) are web-based systems that allow instructors and/or students to share materials, submit and return assignments, and communicate online. In this study, we explored the uses and perceived benefits of using a LMS to support traditional classroom teaching as reported by instructors and students at a large American Midwestern university. We examined two years of survey data focusing on specific uses of the LMS that emphasized either efficient communication or interactive teaching and learning practices. We matched aggregate user log data with corresponding survey items to see if system use was consistent with patterns seen in the survey results. Findings suggest that instructors and students value tools and activities for efficient communication more than interactive tools for innovating existing practices. However, survey item analysis reveals that instructors and students also highly value the teaching and learning tools within the LMS.     

Analyzing CAD competence with univariate and multivariate learning curve models

Understanding how learning occurs, and what improves or impedes the learning process is of importance to academicians and practitioners; however, empirical research on validating learning curves is sparse. This paper contributes to this line of research by collecting and analyzing CAD (computer-aided design) procedural and cognitive performance data for novice trainees during 16-weeks of training. The declarative performance is measured by time, and the procedural performance by the number of features used to construct a design part. These data were analyzed using declarative or procedural performance separately as predictors (univariate), or a combination of declarative or procedural predictors (multivariate). Furthermore, a method to separate the declarative and procedural components from learning curve data is suggested.     

User utility discovery for priority-based network resource pricing

While equilibrium analysis has been commonly used for network pricing under the assumption that user utility functions are precisely known, many researchers have criticized the validity of the assumption. In this paper, we propose a solution for bridging the gap between the existing theoretical work on optimal pricing and the unavailability of precise user utility information in real networks. In the proposed method, the service provider obtains increasingly more accurate estimates of user utility functions by iteratively changing the prices of service levels and observing the users’ service-level choices under various prices. Our study’s contribution is twofold. First, we have developed a general principle for estimating user utility functions. Especially, we present the utility estimation for dynamic user population. Second, we have developed a method for setting prices that can optimize the extraction of information about user utility functions. The extensive simulation results demonstrate the effectiveness of our method.     

Iterative Learning Control for uncertain systems: Robust monotonic convergence analysis

In this paper, we present a novel Robust Monotonic Convergence (RMC) analysis approach for finite time interval Iterative Learning Control (ILC) for uncertain systems. For that purpose, a finite time interval model for uncertain systems is introduced. This model is subsequently used in an RMC analysis based on μ analysis. As a result, we can handle additive and multiplicative uncertainty models in the RMC problem formulation, analyze RMC of linear time invariant MIMO systems controlled by any linear trial invariant ILC controller, and formulate additional straightforward RMC conditions for ILC controlled systems. To illustrate the derived results, we analyze the RMC properties of linear quadratic (LQ) norm optimal ILC.