Weight smoothing to improve network generalization

A weight smoothing algorithm is proposed in this paper to improve a neural network's generalization capability. The algorithm can be used when the data patterns to be classified are presented on an n-dimensional grid (n>/=1) and there exists some correlations among neighboring data points within a pattern. For a fully-interconnected feedforward net, no such correlation information is embedded into the architecture. Consequently, the correlations can only be extracted through sufficient amount of network training. With the proposed algorithm, a smoothing constraint is incorporated into the objective function of backpropagation to reflect the neighborhood correlations and to seek those solutions that have smooth connection weights. Experiments were performed on problems of waveform classification, multifont alphanumeric character recognition, and handwritten numeral recognition. The results indicate that (1) networks trained with the algorithm do have smooth connection weights, and (2) they generalize better.     

Network information criterion-determining the number of hiddenunits for an artificial neural network model

The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of AIC. This relation leads to a new network information criterion which is useful for selecting the optimal network model based on a given training set.     

Pipelining and bypassing in a VLIW processor

This short note describes issues involved in the bypassing mechanism for a very long instruction word (VLIW) processor and its relation to the pipeline structure of the processor. The authors first describe the pipeline structure of their processor and analyze its performance and compare it to typical RISC-style pipeline structures given the context of a processor with multiple functional units. Next they study the performance effects of various bypassing schemes in terms of their effectiveness in resolving pipeline data hazards and their effect on the processor cycle time     

Second-order bounds on the domain of attraction and the rate ofconvergence of nonlinear dynamical systems and neural networks

We present a method for analyzing the convergence properties of nonlinear dynamical systems yielding second-order bounds on the domain of attraction of an asymptotically stable equilibrium point and on the time of convergence in the estimated domain. We show that under certain conditions on the system, there exists an analytic solution to the corresponding optimization problem. The method is applied in analyzing the dynamics of a neural network model.     

A delta transform approach to loop gain-phase shaping design of robust digital control systems

This paper addresses the existence of loop gain-phase shaping (LGPS) solutions for the design of robust digital control systems for SISO, minimum-phase, continuous-time processes with parametric uncertainty. We develop the frequency response properties of LGPS for discrete-time systems using the Δ-transform, a transform method that applies to both continuous-time and discrete-time systems. A theorem is presented which demonstrates that for reasonable specifications there always exists a sampling period such that the robust digital control problem has a solution. Finally, we offer a procedure for estimating the maximum feasible sampling period for LGPS solutions to robust digital control problems.     

Base-type languages: Program design by upper semantic approximation

Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 140–155, July–August 1994.     

Fault trees—Problems and the modern state of investigations

Published in Kibernetika i Sistemnyi Analiz, No. 3, pp. 128–150, May–June, 1994.