Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification

Mixture of experts (ME) models comprise a family of modular neural network architectures aiming at distilling complex problems into simple subtasks. This is done by deploying a separate gating module for softly dividing the input space into overlapping regions to be each assigned to one or more expert networks. Conversely, support vector machines (SVMs) refer to kernel-based methods, neural-network-alike models that constitute an approximate implementation of the structural risk minimization principle. Such learning machines follow the simple, but powerful idea of nonlinearly mapping input data into high-dimensional feature spaces wherein a linear decision surface discriminating different regions is properly designed. In this work, we formally characterize and empirically evaluate a novel approach, named as Mixture of Support Vector Machine Experts (MSVME), whose main purpose is to combine the complementary properties of both SVM and ME models. In the formal characterization, an algorithm based on a maximum likelihood criterion is considered for the MSVME training, and we demonstrate that it is possible to train each expert based on an SVM perspective. Regarding the empirical evaluation, simulation results involving nonlinear dynamic system identification problems are reported, contrasting the performance shown by the MSVME approach with that exhibited by conventional SVM and ME models.     

Recurrent neuro-controller design for an inverted pendulum using evolution strategy

Using block-pulse functions, a new method of identifying a single-input single-output dynamic system is presented. The input-output record is expressed in a block-pulse series which transforms the differential equation of the dynamic system into a computationally convenient algebraic form. Compared with the Walsh function approach, the proposed method is simpler to compute, is more suitable for computer programming and provides the same accuracy.     

BAIS: A Bayesian Artificial Immune System for the effective handling of building blocks

Significant progress has been made in theory and design of Artificial Immune Systems (AISs) for solving hard problems accurately. However, an aspect not yet widely addressed by the research reported in the literature is the lack of ability of the AISs to deal effectively with building blocks (partial high-quality solutions coded in the antibody). The available AISs present mechanisms for evolving the population that do not take into account the relationship among the variables of the problem, potentially causing the disruption of high-quality partial solutions. This paper proposes a novel AIS with abilities to identify and properly manipulate building blocks in optimization problems. Instead of using cloning and mutation to generate new individuals, our algorithm builds a probabilistic model representing the joint probability distribution of the promising solutions and, subsequently, uses this model for sampling new solutions. The probabilistic model used is a Bayesian network due to its capability of properly capturing the most relevant interactions among the variables. Therefore, our algorithm, called Bayesian Artificial Immune System (BAIS), represents a significant attempt to improve the performance of immune-inspired algorithms when dealing with building blocks, and hence to solve efficiently hard optimization problems with complex interactions among the variables. The performance of BAIS compares favorably with that produced by contenders such as state-of-the-art Estimation of Distribution Algorithms.     

Pattern classification with mixtures of weighted least-squares support vector machine experts

Support Vector Machine (SVM) classifiers are high-performance classification models devised to comply with the structural risk minimization principle and to properly exploit the kernel artifice of nonlinearly mapping input data into high-dimensional feature spaces toward the automatic construction of better discriminating linear decision boundaries. Among several SVM variants, Least-Squares SVMs (LS-SVMs) have gained increased attention recently due mainly to their computationally attractive properties coming as the direct result of applying a modified formulation that makes use of a sum-squared-error cost function jointly with equality, instead of inequality, constraints. In this work, we present a flexible hybrid approach aimed at augmenting the proficiency of LS-SVM classifiers with regard to accuracy/generalization as well as to hyperparameter calibration issues. Such approach, named as Mixtures of Weighted Least-Squares Support Vector Machine Experts, centers around the fusion of the weighted variant of LS-SVMs with Mixtures of Experts models. After the formal characterization of the novel learning framework, simulation results obtained with respect to both binary and multiclass pattern classification problems are reported, ratifying the suitability of the novel hybrid approach in improving the performance issues considered.     

Microstructure and properties of a composite system for dental applications composed of glass-ceramics in the SiO2–Li2O–ZrO2–P2O5 system and ZrO2-ceramic (TZP)

The objective of the study was to develop a biocompatible composite system which was composed of TZP-ceramic (tetragonal zirconia polycrystals, ZrO₂ stabilized with 3 mol% Y₂O₃) and two glass-ceramics of the SiO₂–Li₂O–ZrO₂–P₂O₅ type. The metal-free composite system would satisfy the translucency, the biocompatibility and the strength requirements of dentistry. The two glass-ceramics of the SiO₂–Li₂O–ZrO₂–P₂O₅ type with a content of 15 and 20 wt% ZrO₂ respectively, were chemically and physically adapted to TZP-ceramic. The glass-ceramics were used as a dentin buildup material. The TZP-ceramic had the function of a root post. The shape of the post was cylindrical with a conical tip. The composite system was easy to process through viscous flow of the glass-ceramic at 900 and 1000°C, respectively. The microstructure and the mechanical properties of two glass-ceramics of the SiO₂–Li₂O–ZrO₂–P₂O₅ type were examined therefore.     

Texture and residual strain in two SiC/Ti-6-2-4-2 titanium composites

Residual strain and texture variations were measured in two titanium matrix composites reinforced with silicon carbide fibers (Ti/SiC) of similar composition but fabricated by different processing routes. Each composite comprised a Ti-6242 α/β matrix alloy containing vol 35 pct continuous SiC fibers. In one, the matrix was produced by a plasma sprayed (PS) route, and in the other by a wiredrawn (WD) process. The PS and WD composites were reinforced with SCS-6 (SiC) and Trimarc (SiC) fibers, respectively. The texture in the titanium matrices differed significantly. The titanium matrix for the PS material exhibited random texture pre and post fabrication of the composite. For the WD material, the starting texture of the monolithic titanium matrix was ≈17 times random, but after consolidation into composite form, it was ≈6 times random. No significant differences were noted in the fiber-induced matrix residual strains between the composites prepared by the two procedures. However, the Trimarc (WD) fibers recorded higher (≈1.3 times) compressive strains than the SCS-6 (PS) fibers. Stresses and stress balance results are reported. Plane-specific elastic moduli, measured in load tests on the unreinforced matrices, showed little difference. This article is based on a presentation made in the Symposium “Mechanisms and Mechanics of Composites Fracture” held October 11–15, 1998, at the TMS Fall Meeting in Rosemont, Illinois, under the auspices of the TMS-SMD/ASM-MSCTS Composite Materials Committee.     

A scalable manufacturing process for flexible active‐matrix e‐paper displays

Abstract— A scalable manufacturing process for fabricating active‐matrix backplanes on low‐cost flexible substrates, a key enabler for electronic‐paper displays, is presented. This process is based on solution processing, ink‐jet printing, and laser patterning. A multilayer architecture is employed to enable high aperture ratio and array performance. These backplanes were combined with E Ink electrophoretic media to create high‐performance displays that have high contrast, are bistable, and can be flexed repeatedly to a radius of curvature of 5 mm.     

Mechanizing some advanced refinement concepts

We describe how the HOL theorem prover can be used to check and apply rules of program refinement. The rules are formulated in the refinement calculus, which is a theory of correctness preserving program transformations. We embed a general command notation with a predicate transformer semantics in the logic of the HOL system. Using this embedding, we express and prove rules for data refinement and superposition refinement of initialized loops. Applications of these proof rules to actual program refinements are checked using the HOL system, with the HOL system generating these conditions. We also indicate how the HOL system is used to prove the verification conditions. Thus, the HOL system can provide a complete mechanized environment for proving program refinements.     

Constructive learning neural network applied to identification and control of a fuel-ethanol fermentation process

In the present work, a constructive learning algorithm was employed to design a near-optimal one-hidden layer neural network structure that best approximates the dynamic behavior of a bioprocess. The method determines not only a proper number of hidden neurons but also the particular shape of the activation function for each node. Here, the projection pursuit technique was applied in association with the optimization of the solvability condition, giving rise to a more efficient and accurate computational learning algorithm. As each activation function of a hidden neuron is defined according to the peculiarities of each approximation problem, better rates of convergence are achieved, guiding to parsimonious neural network architectures. The proposed constructive learning algorithm was successfully applied to identify a MIMO bioprocess, providing a multivariable model that was able to describe the complex process dynamics, even in long-range horizon predictions. The resulting identification model was considered as part of a model-based predictive control strategy, producing high-quality performance in closed-loop experiments.     

A neural architecture for visual information processing

We report on the work done at the Institut für Neuroinformatik in Bochum concerning the development of a neural architecture for the information processing of autonomous visually guided systems acting in a natural environment. Since biological systems like our brain are superior to artificial systems in solving such a task, we use findings from neurophysiology and -anatomy as well as psychophysics for defining processing principles and modules that have been implemented on our mobile platform MARVIN. MARVIN is equipped with an active stereo camera system. Our final objective is to define a neural instruction set for early information processing in the sense of a perception for action approach. From the biological paradigm we use principles like active vision, foveation, two-dimensional cortical layers, mapping, and discrete parametric representations in a task-oriented way to solve problems like obstacle avoidance, path planning, scene recognition, tracking, and 3D perception. This paper has the character of an overview of the work done in this field at our institute. Most of the modules presented here were published either in conference proceedings or in journals which will be referenced for a more thorough discussion of each issue.