Training parsers by inverse reinforcement learning

One major idea in structured prediction is to assume that the predictor computes its output by finding the maximum of a score function. The training of such a predictor can then be cast as the problem of finding weights of the score function so that the output of the predictor on the inputs matches the corresponding structured labels on the training set. A similar problem is studied in inverse reinforcement learning (IRL) where one is given an environment and a set of trajectories and the problem is to find a reward function such that an agent acting optimally with respect to the reward function would follow trajectories that match those in the training set. In this paper we show how IRL algorithms can be applied to structured prediction, in particular to parser training. We present a number of recent incremental IRL algorithms in a unified framework and map them to parser training algorithms. This allows us to recover some existing parser training algorithms, as well as to obtain a new one. The resulting algorithms are compared in terms of their sensitivity to the choice of various parameters and generalization ability on the Penn Treebank WSJ corpus.     

Coinductive big-step operational semantics

Using a call-by-value functional language as an example, this article illustrates the use of coinductive definitions and proofs in big-step operational semantics, enabling it to describe diverging evaluations in addition to terminating evaluations. We formalize the connections between the coinductive big-step semantics and the standard small-step semantics, proving that both semantics are equivalent. We then study the use of coinductive big-step semantics in proofs of type soundness and proofs of semantic preservation for compilers. A methodological originality of this paper is that all results have been proved using the Coq proof assistant. We explain the proof-theoretic presentation of coinductive definitions and proofs offered by Coq, and show that it facilitates the discovery and the presentation of the results.     

A formal analysis of database replication protocols with SI replicas and crash failures

This paper provides a formal specification and proof of correctness of a basic Generalized Snapshot Isolation certification-based data replication protocol for database middleware architectures. It has been modeled using a state transition system, as well as the main system components, allowing a perfect match with the usual deployment in a middleware system. The proof encompasses both safety and liveness properties, as it is commonly done for a distributed algorithm. Furthermore, a crash failure model has been assumed for the correctness proof, although recovery analysis is not the aim of this paper. This allows an easy extension toward a crash-recovery model support in future works. The liveness proof focuses in the uniform commit: if a site has committed a transaction, the rest of sites will either commit it or it would have crashed.     

UAVs Integration in the SWIM Based Architecture for ATM

The System Wide Information Management (SWIM) approach has been conceived to overcome the capacity and flexibility limitations of the current ATM systems. On the other hand the commercial applications of Unmanned Aerial Vehicles (UAVs) require the integration of these vehicles in the ATM. From this perspective, the unavoidable modernization of the ATM is seen as an opportunity to integrate the UAVs with the rest of the air traffic. This paper is devoted to study the feasibility and impact of the aggregation of UAVs on the future ATM supported by a SWIM inspired architecture. Departing from the existing technical documents that describe the fundamentals of SWIM we have explored the compatibility with a potential UAVs integration and also explored how the UAVs could help to improve the future ATM system. We will use the weather application as an example in both cases.     

Toward a human-like biped robot with compliant legs

Conventional models of bipedal walking generally assume rigid body structures, while elastic material properties seem to play an essential role in nature. On the basis of a novel theoretical model of bipedal walking, this paper investigates a model of biped robot which makes use of minimum control and elastic passive joints inspired from the structures of biological systems. The model is evaluated in simulation and a physical robotic platform by analyzing the kinematics and ground reaction force. The experimental results show that, with a proper leg design of passive dynamics and elasticity, an attractor state of human-like walking gait patterns can be achieved through extremely simple control without sensory feedback. The detailed analysis also explains how the dynamic human-like gait can contribute to adaptive biped walking.     

Pattern classification with mixtures of weighted least-squares support vector machine experts

Support Vector Machine (SVM) classifiers are high-performance classification models devised to comply with the structural risk minimization principle and to properly exploit the kernel artifice of nonlinearly mapping input data into high-dimensional feature spaces toward the automatic construction of better discriminating linear decision boundaries. Among several SVM variants, Least-Squares SVMs (LS-SVMs) have gained increased attention recently due mainly to their computationally attractive properties coming as the direct result of applying a modified formulation that makes use of a sum-squared-error cost function jointly with equality, instead of inequality, constraints. In this work, we present a flexible hybrid approach aimed at augmenting the proficiency of LS-SVM classifiers with regard to accuracy/generalization as well as to hyperparameter calibration issues. Such approach, named as Mixtures of Weighted Least-Squares Support Vector Machine Experts, centers around the fusion of the weighted variant of LS-SVMs with Mixtures of Experts models. After the formal characterization of the novel learning framework, simulation results obtained with respect to both binary and multiclass pattern classification problems are reported, ratifying the suitability of the novel hybrid approach in improving the performance issues considered.     

The behaviour of the multi-layer perceptron and the support vector regression learning methods in the prediction of NO and NO2 concentrations in Szeged, Hungary

The main aim of this paper is to predict NO and NO₂ concentrations 4 days in advance by comparing two artificial intelligence learning methods, namely, multi-layer perceptron and support vector machines, on two kinds of spatial embedding of the temporal time series. Hourly values of NO and NO₂ concentrations, as well as meteorological variables were recorded in a cross-road monitoring station with heavy traffic in Szeged, in order to build a model for predicting NO and NO₂ concentrations several hours in advance. The prediction of NO and NO₂ concentrations was performed partly on the basis of their past values, and partly on the basis of temperature, humidity and wind speed data. Since NO can be predicted more accurately, its values were considered primarily when forecasting NO₂. Time series prediction can be interpreted in a way that is suitable for artificial intelligence learning. Two effective learning methods, namely, multi-layer perceptron and support vector regression are used to provide efficient non-linear models for NO and NO₂ time series predictions. Multi-layer perceptron is widely used to predict these time series, but support vector regression has not yet been applied for predicting NO and NO₂ concentrations. Three commonly used linear algorithms were considered as references: 1-day persistence, average of several day persistence and linear regression. Based on the good results of the average of several day persistence, a prediction scheme was introduced, which forms weighted averages instead of simple ones. The optimization of these weights was performed with linear regression in linear case and with the learning methods mentioned in non-linear case. Concerning the NO predictions, the non-linear learning methods give significantly better predictions than the reference linear methods. In the case of NO₂, the improvement of the prediction is considerable, however, it is less notable than for NO.     

An integrated domain analysis approach for teleoperated systems

Teleoperated systems for ship hull maintenance (TOS) are robotic systems for ship maintenance tasks, such as cleaning or painting a ship’s hull. The product line paradigm has recently been applied to TOS, and a TOS reference architecture has thus been designed. However, TOS requirements specifications have not been developed in any rigorous way with reuse in mind. We therefore believe that an opportunity exists to increase the abstraction level at which stakeholders can reason about this product line. This paper reports an experience in which this TOS domain was analyzed, including the lessons learned in the construction and use of the TOS domain model. The experience is based on the application of extensions of well-known domain analysis techniques, together with the use of quality attribute templates traced to a feature model to deal with non-functional issues. A qualitative research method (action research) was used to carry out the experience.     

Reinforcement learning of pedagogical policies in adaptive and intelligent educational systems

In an adaptive and intelligent educational system (AIES), the process of learning pedagogical policies according the students needs fits as a Reinforcement Learning (RL) problem. Previous works have demonstrated that a great amount of experience is needed in order for the system to learn to teach properly, so applying RL to the AIES from scratch is unfeasible. Other works have previously demonstrated in a theoretical way that seeding the AIES with an initial value function learned with simulated students reduce the experience required to learn an accurate pedagogical policy. In this paper we present empirical results demonstrating that a value function learned with simulated students can provide the AIES with a very accurate initial pedagogical policy. The evaluation is based on the interaction of more than 70 Computer Science undergraduate students, and demonstrates that an efficient and useful guide through the contents of the educational system is obtained.