Dynamics of learning near singularities in layered networks

We explicitly analyze the trajectories of learning near singularities in hierarchical networks, such as multilayer perceptrons and radial basis function networks, which include permutation symmetry of hidden nodes, and show their general properties. Such symmetry induces singularities in their parameter space, where the Fisher information matrix degenerates and odd learning behaviors, especially the existence of plateaus in gradient descent learning, arise due to the geometric structure of singularity. We plot dynamic vector fields to demonstrate the universal trajectories of learning near singularities. The singularity induces two types of plateaus, the on-singularity plateau and the near-singularity plateau, depending on the stability of the singularity and the initial parameters of learning. The results presented in this letter are universally applicable to a wide class of hierarchical models. Detailed stability analysis of the dynamics of learning in radial basis function networks and multilayer perceptrons will be presented in separate work.     

Modeling basal ganglia for understanding Parkinsonian reaching movements

We present a computational model that highlights the role of basal ganglia (BG) in generating simple reaching movements. The model is cast within the reinforcement learning (RL) framework with correspondence between RL components and neuroanatomy as follows: dopamine signal of substantia nigra pars compacta as the temporal difference error, striatum as the substrate for the critic, and the motor cortex as the actor. A key feature of this neurobiological interpretation is our hypothesis that the indirect pathway is the explorer. Chaotic activity, originating from the indirect pathway part of the model, drives the wandering, exploratory movements of the arm. Thus, the direct pathway subserves exploitation, while the indirect pathway subserves exploration. The motor cortex becomes more and more independent of the corrective influence of BG as training progresses. Reaching trajectories show diminishing variability with training. Reaching movements associated with Parkinson's disease (PD) are simulated by reducing dopamine and degrading the complexity of indirect pathway dynamics by switching it from chaotic to periodic behavior. Under the simulated PD conditions, the arm exhibits PD motor symptoms like tremor, bradykinesia and undershooting. The model echoes the notion that PD is a dynamical disease.     

Population coding and decoding in a neural field: a computational study

This study uses a neural field model to investigate computational aspects of population coding and decoding when the stimulus is a single variable. A general prototype model for the encoding process is proposed, in which neural responses are correlated, with strength specified by a gaussian function of their difference in preferred stimuli. Based on the model, we study the effect of correlation on the Fisher information, compare the performances of three decoding methods that differ in the amount of encoding information being used, and investigate the implementation of the three methods by using a recurrent network. This study not only rediscovers main results in existing literatures in a unified way, but also reveals important new features, especially when the neural correlation is strong. As the neural correlation of firing becomes larger, the Fisher information decreases drastically. We confirm that as the width of correlation increases, the Fisher information saturates and no longer increases in proportion to the number of neurons. However, we prove that as the width increases further--wider than (sqrt)2 times the effective width of the turning function--the Fisher information increases again, and it increases without limit in proportion to the number of neurons. Furthermore, we clarify the asymptotic efficiency of the maximum likelihood inference (MLI) type of decoding methods for correlated neural signals. It shows that when the correlation covers a nonlocal range of population (excepting the uniform correlation and when the noise is extremely small), the MLI type of method, whose decoding error satisfies the Cauchy-type distribution, is not asymptotically efficient. This implies that the variance is no longer adequate to measure decoding accuracy.     

Conformal Transformation of Kernel Functions: A Data-Dependent Way to Improve Support Vector Machine Classifiers

In this paper we extend the conformal method of modifying a kernel function to improve the performance of Support Vector Machine classifiers [14, 15]. The kernel function is conformally transformed in a data-dependent way by using the information of Support Vectors obtained in primary training. We further investigate the performances of modified Gaussian Radial Basis Function and Polynomial kernels. Simulation results for two artificial data sets show that the method is very effective, especially for correcting bad kernels. This revised version was published online in August 2006 with corrections to the Cover Date.     

Synchronous firing and higher-order interactions in neuron pool

The stochastic mechanism of synchronous firing in a population of neurons is studied from the point of view of information geometry. Higher-order interactions of neurons, which cannot be reduced to pairwise correlations, are proved to exist in synchronous firing. In a neuron pool where each neuron fires stochastically, the probability distribution q(r) of the activity r, which is the fraction of firing neurons in the pool, is studied. When q(r) has a widespread distribution, in particular, when q(r) has two peaks, the neurons fire synchronously at one time and are quiescent at other times. The mechanism of generating such a probability distribution is interesting because the activity r is concentrated on its mean value when each neuron fires independently, because of the law of large numbers. Even when pairwise interactions, or third-order interactions, exist, the concentration is not resolved. This shows that higher-order interactions are necessary to generate widespread activity distributions. We analyze a simple model in which neurons receive common overlapping inputs and prove that such a model can have a widespread distribution of activity, generating higher-order stochastic interactions.     

Gradient Adaptive Paraunitary Filter Banks for Spatio-Temporal Subspace Analysis and Multichannel Blind Deconvolution

Paraunitary filter banks are important for several signal processing tasks, including coding, multichannel deconvolution and equalization, adaptive beamforming, and subspace processing. In this paper, we consider the task of adapting the impulse response of a multichannel paraunitary filter bank via gradient ascent or descent on a chosen cost function. Our methods are spatio-temporal generalizations of gradient techniques on the Grassmann and Stiefel manifolds, and we prove that they inherently maintain the paraunitariness of the multichannel adaptive system over time. We then discuss the necessary practical approximations, modifications, and simplifications of the methods for solving two relevant signal processing tasks: (i) spatio-temporal subspace analysis and (ii) multichannel blind deconvolution. Simulations indicate that our methods can provide simple, useful solutions to these important problems.     

Ductility and strain-induced transformation in a high-strength transformation-induced plasticity-aided dual-phase steel

The influence of forming temperature and strain rate on the ductility and strain-induced transformation behavior of retained austenite in a ferritic 0.4C-1.5Si-1.5Mn (wt pct) dual-phase steel containing fine retained austenite islands of about 15 vol pct has been investigated. Ex- cellent combinations of total elongations (TELs), about 48 pct, and tensile strength (TS), about 1000 MPa, were obtained at temperatures between 100 °C and 200 °C and at a strain rate of 2.8 X 10_-4/s. Under these optimum forming conditions, the flow curves were characterized by intensive serrations and increased strain-hardening rate over a large strain range. The retained austenite islands were mechanically the most stable at temperatures between 100 °C and 200 °C, and the retained austenite stability appeared to be mainly controlled by strain-induced martensite and bainite transformations (SIMT and SIBT, respectively), with deformation twinning occur- ring in the retained austenite. The enhanced TEL and forming temperature dependence of TEL were primarily connected with both the strain-induced transformation behavior and retained aus- tenite stability.     

Blind source separation-semiparametric statistical approach

The semiparametric statistical model is used to formulate the problem of blind source separation. The method of estimating functions is applied to this problem. It is shown that an estimator of the mixing matrix or its learning version can be described in terms of an estimating function. The statistical efficiencies of these algorithms are studied. The main results are as follows. (1) The space consisting of all the estimating functions is derived. (2) The space is decomposed into the orthogonal sum of the admissible part and a redundant ancillary part. For any estimating function, one can find a better or equally good estimator in the admissible part. (3) The Fisher efficient (that is, asymptotically best) estimating functions are derived. (4) The stability of learning algorithms is studied     

Synthesis and viscoelastic properties of new thermosetting resins having isocyanurate and oxazolidone rings in their molecular structures

A new type of thermosetting resin can be obtained from solventless varnishes composed of polyfunctional isocyanates and epoxides. The IR spectra show that the cured resins have isocyanurate and oxazolidone rings in their chemical structure, hence their name, isocyanurate–oxazolidone resins. The viscoelastic behavior of the cured resins can be varied considerably by selecting an appropriate reactant species and changing the equivalent ratio of the isocyanate and epoxy groups in the varnishes. Three relaxations, labeled α, β, and γ, characterize their dynamic mechanical spectra. The α-dispersion is due to the glass transition, the β-dispersion, to local mode motions of the backbone chains of resins, and the γ-dispersion, to the molecular motions of long alkyl chains. If certain additives are mixed into the varnishes before curing, microphase separation takes place during the reactions. The resins finally obtained have a peculiar physical structure in which rubber particles are dispersed throughout a matrix phase as confirmed by scanning electron-microscopic analysis and viscoelastic behavior.