Neural networks and non-linear statistical methods: an application to the modelling of price–quality relationships

This paper examines the potential of a neural network (NN) approach to the analysis of ‘hedonic’ regressions, in which price is dependent on quality characteristics. The aim of the regressions is to measure, using objective data, the valuation consumers place on these characteristics. A neural network approach is employed because of potential non-linearities in the hedonic functions, using the property of ‘universal approximation’. Our NN implementation goes beyond the now-orthodox approach in using the Polytope algorithm, which we compare with Backpropagation, and uses two hidden layers. The results obtained provide an improvement on linear formulations, but the improvement in this case is relatively marginal. We view NN modelling as a useful means of specification testing and hence our results imply some support for a linear formulation as an adequate approximation. From a managerial perspective, the linear model is more easily interpreted. NN modelling is potentially very time-consuming, especially with the Polytope algorithm, and requires a good deal of technical skill.Scope and purposeThe application area studied in the paper involves ‘hedonic’ regressions, which is the term usually employed for regressions of prices on the characteristics of goods. These employ objective data rather than subjective evaluations of intent and as well as having predictive capacity they serve to indicate the valuation placed on characteristics by consumers. In the extensive literature on the subject there is extensive debate on the appropriate functional form for the regressions. We have employed neural networks (NNs) in order to cast some light on the issue, because of their property of ‘universal approximation’ which, although in danger of being taken too literally means a capacity to ‘mimic’ a wide variety of shapes. We specifically employ an NN model as a test of linearity for hedonic functions, using the Polytope algorithm as an alternative to the standard Backpropagation method. Our results indicate that only a marginal improvement in goodness of fit is obtained, despite various theoretical arguments against a linear formulation. The linear model is given some support in our work as an adequate working approximation.     

On System Functions with the Property of Separability†

This paper considers linear differential (time-varying) systems which may be described by either of two system functions based on a specified integral transform. In particular, those systems are discussed for which at least one of the aforementioned system functions is separable in its two arguments. Physical interpretations of separable system functions are given and two theorems are proved which yield sufficient conditions for the presence of this property. It is also proved that the so-called ‘bi-frequency’ function of Zadeh must be Separable for linear differential systems. Finally, the problem of approximately representing a given system by a separable system function based on the Laplace transform is discussed.     

Non-parametric identification of continuous-time Hammerstein systems using Gaussian process model and particle swarm optimization

This paper deals with a non-parametric identification of continuous-time Hammerstein systems using Gaussian process (GP) models. A Hammerstein system consists of a memoryless non-linear static part followed by a linear dynamic part. The identification model is derived using the GP prior model which is described by the mean function vector and the covariance matrix. This prior model is trained by the separable least-squares (LS) approach combining the linear LS method with particle swarm optimization to minimize the negative log marginal likelihood of the identification data. Then the non-linear static part is estimated by the predictive mean function of the GP, and the confidence measure of the estimated non-linear static part is evaluated by the predictive covariance function of the GP. Simulation results are shown to illustrate the proposed method.     

Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks

This study deals with the neuro-fuzzy (NF) modelling of a real industrial winding process in which the acquired NF model can be exploited to improve control performance and achieve a robust fault-tolerant system. A new simulator model is proposed for a winding process using non-linear identification based on a recurrent local linear neuro-fuzzy (RLLNF) network trained by local linear model tree (LOLIMOT), which is an incremental tree-based learning algorithm. The proposed NF models are compared with other known intelligent identifiers, namely multilayer perceptron (MLP) and radial basis function (RBF). Comparison of our proposed non-linear models and associated models obtained through the least square error (LSE) technique (the optimal modelling method for linear systems) confirms that the winding process is a non-linear system. Experimental results show the effectiveness of our proposed NF modelling approach.     

Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks

This study deals with the neuro-fuzzy (NF) modelling of a real industrial winding process in which the acquired NF model can be exploited to improve control performance and achieve a robust fault-tolerant system. A new simulator model is proposed for a winding process using non-linear identification based on a recurrent local linear neuro-fuzzy (RLLNF) network trained by local linear model tree (LOLIMOT), which is an incremental tree-based learning algorithm. The proposed NF models are compared with other known intelligent identifiers, namely multilayer perceptron (MLP) and radial basis function (RBF). Comparison of our proposed non-linear models and associated models obtained through the least square error (LSE) technique (the optimal modelling method for linear systems) confirms that the winding process is a non-linear system. Experimental results show the effectiveness of our proposed NF modelling approach.     

Converse Schwarz inequality and an instability-related property of feedback systems

The converse of Schwarz inequality is applied to the analysis of an instability-related property (hereafter called Property P) of feedback systems. For systems containing a linear time invariant part with a non-linear (time varying) gain in the feedback path, the conditions for the Property P to hold are given ill ‘ multiplier ’ form; for more general systems, these conditions are expressed in terms of the passivity and finite gains of certain operators derived from the system (forward and feedback) operators.     

Correlation analysis method based SISO neuro-fuzzy Wiener model

A novel identification algorithm for neuro-fuzzy based single-input-single-output (SISO) Wiener model with colored noises is presented in this paper. The separable signal is adopted to identify the Wiener model, leading to the identification problem of the linear part separated from nonlinear counterpart. Then, the correlation analysis method can be employed for identification of linear part. Moreover, in the presence of random signal, the least square method based parameters estimation algorithm of static nonlinear part are proposed to avoid the impact of colored noise. As a result, proposed method can circumvent the problem of initialization and convergence of the model parameters encountered by the existing iterative algorithms used for identification of Wiener model. Examples are used to verify the effectiveness of the proposed method.     

基于神经网络的多变量非线性自适应解耦控制研究

提出神经网络前馈自适应解耦控制算法．该算法将多变量非线性系统在平衡点处利用Taylor公式展开．分为线性部分和高阶非线性部分。这样．将高阶非线性部分的影响视为可测干扰，采用前馈补偿的方法加以消除．就可以借助多变量线性系统的自适应解耦控制算法．实现多变量非线性系统的自适应解耦控制．这种方法可以取消被解耦系统为最小相位的限制。     

An O(n) algorithm for the linear multiple choice knapsack problem and related problems

We present an O(n) algorithm for the Linear Multiple Choice Knapsack Problem and its d-dimensional generalization which is based on Megiddo's (1982) algorithm for linear programming. We also consider a certain type of convex programming problems which are common in geometric location models. An application of the linear case is an O(n) algorithm for finding a least distance hyperplane in R^d according to the rectilinear norm. The best previously available algorithm for this problem was an O(n log²n) algorithm for the two-dimensional case. A simple application of the nonlinear case is an O(n) algorithm for finding the point at which a ‘pursuer’ minimizes its distance from the furthest among n ‘targets’, when the trajectories involved are straight lines in R^d.     

Stable decentralized adaptive control design of robot manipulators using neural network approximations

In this paper, we present a decentralized neural network (NN) adaptive technique for control of robot manipulators in the presence of unknown non-linear functions. Radial basis function NNs are used to approximate the non-linear functions to include the case of both parametric and dynamic uncertainty in each subsystem. The robustifying terms are added to the controllers to overcome the effects of the interconnections. The stability can be guaranteed by using a rigid proof. Finally, simulation is given to illustrate the effectiveness of the proposed algorithm.     

一种湿度传感器温度补偿的融合算法

针对自动气象站上湿度传感器在实际应用过程中易受温度影响的问题,提出采用RBF神经网络与最小二乘相结合的融合算法实现湿度传感器的温度补偿。该方法将湿度传感器在温度影响下的特性曲线分为两个非线性段和一个线性段,并且自适应的确定线性段和非线性段,在线性段利用最小二乘方法拟合出直线方程,在非线性段利用RBF神经网络补偿温度产生的影响。仿真结果表明,这种方法简单易行,与一般的BP神经网络和最小二乘多项式方法相比,具有拟合训练速度快,补偿精度高的特点,可以有效用于湿度传感器的温度补偿,提高传感器的测量精度和可靠性。     

An iterative algorithm learning the maximal margin classifier

A simple learning algorithm for maximal margin classifiers (also support vector machines with quadratic cost function) is proposed. We build our iterative algorithm on top of the Schlesinger-Kozinec algorithm (S-K-algorithm) from 1981 which finds a maximal margin hyperplane with a given precision for separable data. We suggest a generalization of the S-K-algorithm (i) to the non-linear case using kernel functions and (ii) for non-separable data. The requirement in memory storage is linear to the data. This property allows the proposed algorithm to be used for large training problems.The resulting algorithm is simple to implement and as the experiments showed competitive to the state-of-the-art algorithms. The implementation of the algorithm in Matlab is available. We tested the algorithm on the problem aiming at recognition poor quality numerals.     

Modelling of the inventory replenishment problem with a two-segment piecewise linear demand

The classical inventory replenishment problem with a linear function in demand uses a ‘single-segment’ linear function as its demand and can be modelled by a simple algorithm. Moreover, this article extends the algorithm to provide a heuristic solution for the inventory replenishment model with a two-segment linear function in demand called the ‘two-segment piecewise linear demand model’. In addition, this article proposes a general procedure for solving both models. Meanwhile, several examples taken from the literature illustrate our algorithm for these two models with convincing results. Furthermore, this study shows that when the demand is a two-segment piecewise linear function over time, it is better to use the proposed algorithm rather than devising a decoupled solution approach by treating segments separately. Finally, a sensitivity analysis of two factors, demand and cost, is performed. The model is highly extensible and applicable, so it can serve as an inventory planning tool to solve the replenishment problem.     

一种基于混合模型的实时网络流量预测算法

流量预测是流量工程、拥塞控制和网络管理的核心问题.网络流量由大量的非线性变化部分和少量的但不可忽略的线性变化部分组成.现有的网络流量预测算法只是单一采用线性或者非线性的方法进行处理,这种片面性造成预测的准确度和实时性难以保证.针对网络流量的特点,提出了一种基于卡尔曼滤波和小波分析混合的流量预测算法.通过对网络流量的线性部分和非线性部分进行区分对待,从而提高预测的准确度和实时性.仿真结果表明,该算法与单一的线性预测算法和非线性预测算法相比,具有较高的预测精度和较好的实时性.     

Multi-model control of MIMO systems: location and control algorithms

This paper presents a multi-model control scheme that depends on the multiple representation of a process using linear models. A dynamic system can be represented by several models, each of which is different in either the simplifications involved, the reductions involved, or the dynamic characteristics. A new tracking multi-model control algorithm for deterministic systems is proposed. An auxiliary input called the ‘state correction’ is calculated and applied to the models so as to minimize a performance index which is a function of the difference between the process outputs and the model outputs. A simulation study is given to show the potential of the proposed algorithm.     

Stability and stabilizability of fuzzy-neural-linear controlsystems

This paper discusses stability analysis of fuzzy-neural-linear (FNL) control systems which consist of combinations of fuzzy models, neural network (NN) models, and linear models. The authors consider a relation among the dynamics of NN models, those of fuzzy models and those of linear models. It is pointed out that the dynamics of linear models and NN models can be perfectly represented by Takagi-Sugeno (T-S) fuzzy models whose consequent parts are described by linear equations. In particular, the authors present a procedure for representing the dynamics of NN models via T-S fuzzy models. Next, the authors recall stability conditions for ensuring stability of fuzzy control systems in the sense of Lyapunov. The stability criteria is reduced to the problem of finding a common Lyapunov function for a set of Lyapunov inequalities. The stability conditions are employed to analyze stability of FNL control systems. Finally, stability analysis for four types of FNL control systems is demonstrated     

Decoupling with dynamic compensation for strong invertible affine non-linear systems

In this paper we tackle, for affine non-linear systems, the ‘Morgan Problem’, i.e. the scalar-input-scalar-output decoupling problem for square systems, with dynamic compensation. The result provided here (Theorem 3.1) generalizes that one previously given by Wang (1970) for linear systems.     

An iterative Kalman smoother/least-squares algorithm for the identification of delta-ARX models

Additive measurement noise on the output signal is a significant problem in the δ-domain and disrupts parameter estimation of auto-regressive exogenous (ARX) models. This article deals with the identification of δ-domain linear time-invariant models of ARX structure (i.e. driven by known input signals and additive process noise) by using an iterative identification scheme, where the output is also corrupted by additive measurement noise. The identification proceeds by mapping the ARX model into a canonical state-space framework, where the states are the measurement noise-free values of the underlying variables. A consequence of this mapping is that the original parameter estimation task becomes one of both a state and parameter estimation problem. The algorithm steps between state estimation using a Kalman smoother and parameter estimation using least squares. This approach is advantageous as it avoids directly differencing the noise-corrupted ‘raw’ signals for use in the estimation phase and uses different techniques to the common parametric low-pass filters in the literature. Results of the algorithm applied to a simulation test problem as well as a real-world problem are given, and show that the algorithm converges quite rapidly and with accurate results.     

On-line structure selection for multivariable state-space models

An algorithm is described for the selection of model structure for identifying state-space models of ‘black box’ character. The algorithm receives as ‘input’ a given system in a given parametrization. It is then tested whether this parametrization is suitable (well conditioned) for identification purposes. If not, a better one is selected and the transformation of the system to the new representation is performed. This algorithm can be used as a block both in an iterative, off-line identification procedure, and for recursive, on-line identification. It can be called whenever there is some indication that the model structure is ill-conditioned. It is discussed how the model structure selection algorithm can be interfaced with an off-line identification procedure. A complete procedure is described and tested on real and simulated data.     

Reply to ‘Feasibility of neural network approach in spectral mixture analysis of reflectance spectra’

Neural Networks (NN) have proliferated during recent years, and are widely used in the scientific environment, particularly providing interpretation of results acquired by spectroscopic techniques. Separately and independently, these results were historically analysed and interpreted with ‘classical techniques’, derived from statistical formulations. The purpose of this reply is to analyse under what conditions NN methods have a better performance than the statistical methods, when it is necessary to process a spectrum obtained by a linear spectroscopic technique. The use of Neural Networks methods instead of purely statistical methods for linear spectra analysis and interpretation is discussed.