Stochastic choice of basis functions in adaptive functionapproximation and the functional-link net

A theoretical justification for the random vector version of the functional-link (RVFL) net is presented in this paper, based on a general approach to adaptive function approximation. The approach consists of formulating a limit-integral representation of the function to be approximated and subsequently evaluating that integral with the Monte-Carlo method. Two main results are: (1) the RVFL is a universal approximator for continuous functions on bounded finite dimensional sets, and (2) the RVFL is an efficient universal approximator with the rate of approximation error convergence to zero of order O(C/ radicaln), where n is number of basis functions and with C independent of n. Similar results are also obtained for neural nets with hidden nodes implemented as products of univariate functions or radial basis functions. Some possible ways of enhancing the accuracy of multivariate function approximations are discussed.     

Controlled stochastic model of a communication system with multiplesources

A stochastic model of buffering in a data communication system is considered, with source and sink transmission parameters depending on the number of active sources. For models in this control setting the authors analyze an effective numerical method for evaluating the equilibrium distribution of buffer content. The theoretical basis of the method is established first. Then, it is shown that the method has the same complexity, in terms of the total number of sources, as known analytical methods for the model with constant parameters. Asymptotics for tail probabilities at high buffer levels and under heavy load are also derived, and the complexity of their computation is compared with that of evaluating explicit formulas. In comparison to earlier results this approach reduces the complexity of computing the probability of overflow and its asymptotic estimates. The speed-up stems from the application of interpolation schemes     

The ensemble approach to neural-network learning and generalization

A method is suggested for learning and generalization with a general one-hidden layer feedforward neural network. This scheme encompasses the use of a linear combination of heterogeneous nodes having randomly prescribed parameter values. The learning of the parameters is realized through adaptive stochastic optimization using a generalization data set. The learning of the linear coefficients in the linear combination of nodes is achieved with a linear regression method using data from the training set. One node is learned at a time. The method allows for choosing the proper number of net nodes, and is computationally efficient. The method was tested on mathematical examples and real problems from materials science and technology.     

Comments on "Stochastic choice of basis functions in adaptive function approximation and the functional-link net" [and reply].

This paper includes some comments and amendments of the above-mentioned paper by Igelnik et al. (1995). Subsequently, Theorem 1 in the above-mentioned paper has been revised. The significant change of the original theorem is the space of the thresholds in the hidden layer. The revised theorem says that the thresholds of hidden b(0), should be -w(0).y(0)-u(0), where w(0)=alphaw;(0); w;(0)=(w; (01), ..., y(0d)), and u(0 ) be independent and uniformly distributed in V(d)=[0; Omega]x[-Omega; Omega](d-1), I(d), and [-2dOmega, 2dOmega], respectively. In reply, Igelnik et al. acknowledge that a factor of two was omitted in the statement of a trigonometric identity. However, the validity of the essential point of Theorem 1 is unaltered.     

Kolmogorov''s spline network

In this paper, an innovative neural-network architecture is proposed and elucidated. This architecture, based on the Kolmogorov's superposition theorem (1957) and called the Kolmogorov's spline network (KSN), utilizes more degrees of adaptation to data than currently used neural-network architectures (NNAs). By using cubic spline technique of approximation, both for activation and internal functions, more efficient approximation of multivariate functions can be achieved. The bound on approximation error and number of adjustable parameters, derived in this paper, favorably compares KSN with other one-hidden layer feedforward NNAs. The training of KSN, using the ensemble approach and the ensemble multinet, is described. A new explicit algorithm for constructing cubic splines is presented.     

A net with complex weights

In this article a new neural-network architecture suitable for learning and generalization is discussed and developed. Although similar to the radial basis function (RBF) net, our computational model called the net with complex weights (CWN) has demonstrated a considerable gain in performance and efficiency in number of applications compared to RBF net. Its better performance in classification tasks is explained by the cross-product terms in internal representation of its basis function introduced parsimoniously. Implementation of CWN by the ensemble approach is described. A number of examples, solved using CWN and other networks, are used to illustrate the desirable characteristics of CWN.     

Neural-net computing for interpretation of semiconductor filmoptical ellipsometry parameters

Optical ellipsometry has been found to be a promising technique for monitoring process parameters, such as film composition and film thickness, of semiconductor wafers grown with molecular beam epitaxy. Whereas it is a straightforward task to calculate ellipsometry angles given the thickness of the film and the refractive indexes of the film and substrate, it is a difficult task to invert that mathematical relationship. However, the process must be inverted if the measured parameters are to be interpreted meaningfully in terms of film composition and film thickness. This paper reports on the use of neural-net computing for the inverse mapping of measured ellipsometry parameters. We used a functional-link net which is very efficient in function approximation. The advantage of using the net, however, is not only its speed, but also because some other net architecture characteristics allow us to perform the task in a holistic manner.