Network information criterion-determining the number of hiddenunits for an artificial neural network model

The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of AIC. This relation leads to a new network information criterion which is useful for selecting the optimal network model based on a given training set.     

Addendum to: generic rules to evaluate system-failure frequency

In the paper on Generic rules to evaluate system-failure frequency, (see ibid., vol.49, p.85-7, 2000) the authora did not consider the case of shorter mission times while presenting the rules to evaluate system-failure frequency. Time-specific failure-frequency calculations are required for the systems with shorter mission times. One of the practical uses of time-specific failure-frequency is in finding reasonably accurate estimates of failure-rate and reliability of large systems consisting of repairable components, by using combinatorial methods (without using Markov models). This paper shows that, with some minor modifications, the rules in the original paper for evaluating steady-state failure-frequency can be used to find the time-specific failure-frequency     

On the sensitivity of coupled resonator filters without some direct couplings

This paper presents an analysis of the sensitivity of coupled resonator filters in which some direct couplings are missing. The effect of changes in the coupling coefficients and resonant frequencies of the resonators is investigated by directly computing the gradient of the scattering parameters. It is shown that structures that are modular in the input-to-output direction are much less sensitive than those with modularity in the orthogonal direction for the same frequency response.     

A Novel Band-Reject Element for Pseudoelliptic Bandstop Filters

A novel band-reject element for the design of inline waveguide pseudoelliptic band-reject filters is introduced. The element consists of an offset partial-height post in a rectangular waveguide in which the dominant TE₁₀ mode is propagating. The location of the attenuation pole is primarily determined by the height of the post that generates it. The element allows the implementation of weak, as well as strong coupling coefficients that are encountered in asymmetric band-reject responses with broad stopbands. The coupling strength is controlled by the offset of the post with respect to the center of the main waveguide. The posts are separated by uniform sections of the main waveguide. An equivalent low-pass circuit based on the extracted pole technique is first used in a preliminary design. An improved equivalent low-pass circuit that includes a more accurate equivalent circuit of the band-reject element is then introduced. A synthesis method of the enhanced network is also presented. Filters based on the introduced element are designed, fabricated, and tested. Good agreement between measured and simulated results is achieved     

Dynamics of learning near singularities in layered networks

We explicitly analyze the trajectories of learning near singularities in hierarchical networks, such as multilayer perceptrons and radial basis function networks, which include permutation symmetry of hidden nodes, and show their general properties. Such symmetry induces singularities in their parameter space, where the Fisher information matrix degenerates and odd learning behaviors, especially the existence of plateaus in gradient descent learning, arise due to the geometric structure of singularity. We plot dynamic vector fields to demonstrate the universal trajectories of learning near singularities. The singularity induces two types of plateaus, the on-singularity plateau and the near-singularity plateau, depending on the stability of the singularity and the initial parameters of learning. The results presented in this letter are universally applicable to a wide class of hierarchical models. Detailed stability analysis of the dynamics of learning in radial basis function networks and multilayer perceptrons will be presented in separate work.     

A modified design approach for compact ultra‐wideband microstrip filters

A modified design approach for compact ultra‐wideband microstrip filters with cascaded/folded stepped‐impedance resonators is described. The key feature of the proposed method is to facilitate stronger coupling between stepped‐impedance resonators and, at the same time, eliminate the requirement of extremely small gaps in coupled‐line sections, as found in traditional designs. Simulations and measurements demonstrate that the filters designed with this technique exhibit good reflection, insertion‐loss, and group‐delay performance within the 3.1–10.6 GHz band. © 2009 Wiley Periodicals, Inc. Int J RF and Microwave CAE 2010.     

Modeling basal ganglia for understanding Parkinsonian reaching movements

We present a computational model that highlights the role of basal ganglia (BG) in generating simple reaching movements. The model is cast within the reinforcement learning (RL) framework with correspondence between RL components and neuroanatomy as follows: dopamine signal of substantia nigra pars compacta as the temporal difference error, striatum as the substrate for the critic, and the motor cortex as the actor. A key feature of this neurobiological interpretation is our hypothesis that the indirect pathway is the explorer. Chaotic activity, originating from the indirect pathway part of the model, drives the wandering, exploratory movements of the arm. Thus, the direct pathway subserves exploitation, while the indirect pathway subserves exploration. The motor cortex becomes more and more independent of the corrective influence of BG as training progresses. Reaching trajectories show diminishing variability with training. Reaching movements associated with Parkinson's disease (PD) are simulated by reducing dopamine and degrading the complexity of indirect pathway dynamics by switching it from chaotic to periodic behavior. Under the simulated PD conditions, the arm exhibits PD motor symptoms like tremor, bradykinesia and undershooting. The model echoes the notion that PD is a dynamical disease.     

Integration of stochastic models by minimizing alpha-divergence

When there are a number of stochastic models in the form of probability distributions, one needs to integrate them. Mixtures of distributions are frequently used, but exponential mixtures also provide a good means of integration. This letter proposes a one-parameter family of integration, called alpha-integration, which includes all of these well-known integrations. These are generalizations of various averages of numbers such as arithmetic, geometric, and harmonic averages. There are psychophysical experiments that suggest that alpha-integrations are used in the brain. The alpha-divergence between two distributions is defined, which is a natural generalization of Kullback-Leibler divergence and Hellinger distance, and it is proved that alpha-integration is optimal in the sense of minimizing alpha-divergence. The theory is applied to generalize the mixture of experts and the product of experts to the alpha-mixture of experts. The alpha-predictive distribution is also stated in the Bayesian framework.     

Asymptotic statistical theory of overtraining and cross-validation

A statistical theory for overtraining is proposed. The analysis treats general realizable stochastic neural networks, trained with Kullback-Leibler divergence in the asymptotic case of a large number of training examples. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Based on the cross-validation stopping we consider the ratio the examples should be divided into training and cross-validation sets in order to obtain the optimum performance. Although cross-validated early stopping is useless in the asymptotic region, it surely decreases the generalization error in the nonasymptotic region. Our large scale simulations done on a CM5 are in good agreement with our analytical findings.     

From blind signal extraction to blind instantaneous signal separation: criteria, algorithms, and stability

This paper reports a study on the problem of the blind simultaneous extraction of specific groups of independent components from a linear mixture. This paper first presents a general overview and unification of several information theoretic criteria for the extraction of a single independent component. Then, our contribution fills the theoretical gap that exists between extraction and separation by presenting tools that extend these criteria to allow the simultaneous blind extraction of subsets with an arbitrary number of independent components. In addition, we analyze a family of learning algorithms based on Stiefel manifolds and the natural gradient ascent, present the nonlinear optimal activations (score) functions, and provide new or extended local stability conditions. Finally, we illustrate the performance and features of the proposed approach by computer-simulation experiments.