Comparison of Stochastic Global Optimization Methods to Estimate Neural Network Weights

Training a neural network is a difficult optimization problem because of numerous local minima. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and therefore numerous random restarts with a local algorithm may be more effective than a global algorithm. This study uses Monte-Carlo simulations to determine the efficiency of a local search algorithm relative to nine stochastic global algorithms when using a neural network on function approximation problems. The computational requirements of the global algorithms are several times higher than the local algorithm and there is little gain in using the global algorithms to train neural networks. Since the global algorithms only marginally outperform the local algorithm in obtaining a lower local minimum and they require more computational resources, the results in this study indicate that with respect to the specific algorithms and function approximation problems studied, there is little evidence to show that a global algorithm should be used over a more traditional local optimization routine for training neural networks. Further, neural networks should not be estimated from a single set of starting values whether a global or local optimization method is used.     

An Expression Deformation Approach to Non-rigid 3D Face Recognition

The accuracy of non-rigid 3D face recognition approaches is highly influenced by their capacity to differentiate between the deformations caused by facial expressions from the distinctive geometric attributes that uniquely characterize a 3D face, interpersonal disparities. We present an automatic 3D face recognition approach which can accurately differentiate between expression deformations and interpersonal disparities and hence recognize faces under any facial expression. The patterns of expression deformations are first learnt from training data in PCA eigenvectors. These patterns are then used to morph out the expression deformations. Similarity measures are extracted by matching the morphed 3D faces. PCA is performed in such a way it models only the facial expressions leaving out the interpersonal disparities. The approach was applied on the FRGC v2.0 dataset and superior recognition performance was achieved. The verification rates at 0.001 FAR were 98.35% and 97.73% for scans under neutral and non-neutral expressions, respectively.     

Emotion Recognition in Speech Using Neural Networks

Emotion recognition in speech is a topic on which little research has been done to-date. In this paper, we discuss why emotion recognition in speech is a significant and applicable research topic, and present a system for emotion recognition using one-class- in-one neural networks. By using a large database of phoneme balanced words, our system is speaker- and context-independent. We achieve a recognition rate of approximately 50% when testing eight emotions.     

Prediction of Road Traffic using a Neural Network Approach

A key component of the daily operation and planning activities of a traffic control centre is short-term forecasting, i.e. the prediction of daily to the next few days of traffic flow. Such forecasts have a significant impact on the optimal regulation of the road traffic on all kinds of freeways. They are increasingly important in an environment with increasing road traffic problems. The present paper aims at presenting the effectiveness of a neural network system for prediction based on time-series data. We only use one parameter, namely traffic volume for the forecasting. We employ artificial neural networks for traffic forecasting applied on a road section. Recurrent Jordan networks, popular in the modelling of time series, is examined in this study. Simulation results demonstrate that learning with this type of architecture has a good generalisation ability.     

Higher-Order Neural Networks Applied to 2D and 3D Object Recognition

A higher-order neural network (HONN) can be designed to be invariant to geometric transformations such as scale, translation, and in-plane rotation. Invariances are built directly into the architecture of a HONN and do not need to be learned. Thus, for 2D object recognition, the network needs to be trained on just one view of each object class, not numerous scaled, translated, and rotated views. Because the 2D object recognition task is a component of the 3D object recognition task, built-in 2D invariance also decreases the size of the training set required for 3D object recognition. We present results for 2D object recognition both in simulation and within a robotic vision experiment and for 3D object recognition in simulation. We also compare our method to other approaches and show that HONNs have distinct advantages for position, scale, and rotation-invariant object recognition.The major drawback of HONNs is that the size of the input field is limited due to the memory required for the large number of interconnections in a fully connected network. We present partial connectivity strategies and a coarse-coding technique for overcoming this limitation and increasing the input field to that required by practical object recognition problems.     

A Similarity-Based Aspect-Graph Approach to 3D Object Recognition

This paper describes a view-based method for recognizing 3D objects from 2D images. We employ an aspect-graph structure, where the aspects are not based on the singularities of visual mapping but are instead formed using a notion of shape similarity between views. Specifically, the viewing sphere is endowed with a metric of dis-similarity for each pair of views and the problem of aspect generation is viewed as a segmentation of the viewing sphere into homogeneous regions. The viewing sphere is sampled at regular (5 degree) intervals and the similarity metric is used in an iterative procedure to combine views into aspects with a prototype representing each aspect. This is done in a region-growing regime which stands in contrast to the usual edge detection styles to computing the aspect graph. The aspect growth is constrained such that two aspects of an object remain distinct under the given similarity metric. Once the database of 3D objects is organized as a set of aspects, and prototypes for these aspects for each object, unknown views of database objects are compared with the prototypes and the results are ordered by similarity. We use two similarity metrics for shape, one based on curve matching and the other based on matching shock graphs, which for a database of 64 objects and unknown views of objects from the database give a recall rate of (90.3%, 74.2%, 59.7%) and (95.2%, 69.0%, 57.5%), respectively, for the top three matches; cumulative recall rate based on the top three matches is 98% and 100%, respectively. The result of indexing unknown views of objects not in the database also produce intuitive matches. We also develop a hierarchical indexing scheme to prune unlikely objects at an early stage to improve the efficiency of indexing, resulting in savings of 35% at the top level and of 55% at the next level, cumulatively.     

基于神经网络的静态台标识别系统设计与实现

台标是电视台的重要标志,是对电视节目来源、类型等属性的描述,因此台标识别是实现电视视频自动收录、检索的重要技术基础.针对静态台标的特点和识别中存在的问题,在台标提取和特征识别的基础上,提出了一种基于BP神经网络的台标识别方法,并设计实现了一个静态台标识别系统,对系统的神经网络参数经过反复试验调整,最后得到令人满意的结果.实验结果表明,通过训练和识别分开,对包括半透明台标在内的电视视频,系统具有识别效率高、结果准确等优点.     

一种动态性神经网络的集成方法

提出一种动态性神经网络集成方法,该方法针对若干不同的神经网络,采用加权最近邻技术收集它们的泛化误差信息构成性能矩阵,在此基础上动态选择泛化误差较小的神经网络,经过动态平均形成集成的最终输出结果。实验表明,与其它方法相比,该方法具有令人满意的性能。     

Outdoor Scene Classification by a Neural Tree-Based Approach

This paper describes a Neural Tree (NT) based system for outdoor scene classification. A new NT classifier with backtracking capabilities is employed at different levels of the system architecture. First, it is used to obtain a rough interpretation of the scene by classifying each image pixel into multiple classes of static background objects, e.g. road, sky, vegetation, or into one generic class representing moving objects, e.g. vehicles, pedestrians. Then it is applied to obtain a more accurate scene interpretation by classifying all detected mobile objects into multiple classes, e.g. cars, lorries, buses, and also estimating their pose. Experiments have been performed on a large set of optical and infrared images. System performances are tested on both clean and noisy data, and comparative studies with other classifiers (i.e. a multi-layer perceptron, a binary decision tree, a standard NT and a bank of neural networks) and with other scene classification methods are carried out. Receiveed: 29 May 1998?,Received in revised form: 13 November 1998?Accepted: 15 December 1998     

基于径向基函数神经网络的非线性模型辨识

从径向基函数（RBF）神经网络原理分析出发,提出了一种基于RBF神经网络学习算法,用于对非线性对象模型的拟合与辩识,并将此方法用于实际非线性模型的学习与辩识。结果表明,基于RBF的神经网络可快速完成对样本的学习与拟合,对具有连续特性的线性与非线性模型,具有快速实时的学习速度和优良的学习性能。     

A Neural-Network-Based Approach to Optical Symbol Recognition

In this paper we propose a neural-network-based approach to solving optical symbol recognition problems, from node head recognition to handwritten digit recognition. We demonstrated that node heads could be easily recognized by using a set of fuzzy rules extracted from the parameters of trained neural networks. For handwritten digit recognition we demonstrated that only 12 features are sufficient to achieve a high recognition rate. Several databases were tested to demonstrate the effectiveness and efficiency of the proposed recognition method. This revised version was published online in June 2006 with corrections to the Cover Date.     

基于RBF神经网络的三维残缺数据修补算法

提出了一种基于径向基函数(RBF)神经网络的修补方法,该方法首先通过人工介入法在残缺数据的边界附近获取样本点集,并以其最小二乘拟合平面为基础建立局部坐标系;其次,在此局部坐标系下,将训练后的RBF神经网络仿真曲面用于残缺区域数据点重采;最后,将重采点集通过坐标反变换后,替代原始点云数据中的样本点集。对真实残缺数据进行修补实验,结果表明效果良好。     

一种用于图像目标识别的神经网络及其车型识别应用

构建了一种用于图像目标识别的多层前向神经网络，给出了网络拓扑结构，并成 功地把该神经网络运用到车型识别中。该方法综合了神经网络、模糊逻辑、模式识别的相 关 算法，对图像目标轮廓进行整体识别，达到了较高的目标识别准确率。实践表明，该网 络经 过监督学习后，能摒除图像中一定量干扰像素影响，准确地识别出各种外形车的车型 。     

矢量量化与神经网络相结合的说话人识别系统

介绍了说话人识别系统的基本概念,在分析了传统VQ模型与神经网络模型的基础上,提出了一种VQ与神经网络相结合的说话人识别系统模型。通过提取出的特征参数(MFFC),建立系统模型,实验证明了该模型性能随着时间的变化有较好的稳定性。     

一种适用于模式识别的新型神经网络

提出了一种适用于模式识别的新型神经网络模型——局部有监督特征映射网络,描述了该网络的拓扑结构和学习算法,研究了网络的基本性能,最后将其应用到了质量控制图的模式识别中。理论研究和仿真实验表明,该网络结构简单、算法简洁,收敛速度快、识别精度高,适用于需要大样本训练、随机干扰严重的复杂模式的分类与识别。     

A Glossary of Basic Neural Network Terminology for Regression Problems

Complex jargon represents an impediment to newcomers to the field of neural networks. This document presents a glossary of some of the basic terminology that may be encountered in the literature, with an emphasis on the use of multilayer perceptrons and radial basis functions for regression problems.     

A Neural Network Approach to Predicting Airspeed in Helicopters

A helicopter’s airspeed and sideslip angle is difficult to measure at speeds below 50 knots. This paper describes the application of Artificial Neural Network (ANN) techniques to the helicopter low air-speed problem. Three ANN methods were applied to the problem: a linear network, a Radial Basis Function (RBF) network, and a Multi-Layer Perceptron (MLP), trained using an implementation of the Levenberg–Marquardt (L–M) algorithm. Internally available measurements, such as control positions and body attitudes and rates, were generated using a realistic simulation model of a Lynx helicopter. These measurements formed the inputs to the ANN methods. The MLP was found to be the superior method. Further testing, including a Tagu-chi analysis, indicated the validity of the method. It is concluded that ANN techniques present a promising solution to the helicopter low airspeed problem.     

A Neural Network for Solving Nonlinear Programming Problems

A neural network for solving convex nonlinear programming problems is proposed in this paper. The distinguishing features of the proposed network are that the primal and dual problems can be solved simultaneously, all necessary and sufficient optimality conditions are incorporated, and no penalty parameter is involved. Based on Lyapunov, LaSalle and set stability theories, we prove strictly an important theoretical result that, for an arbitrary initial point, the trajectory of the proposed network does converge to the set of its equilibrium points, regardless of whether a convex nonlinear programming problem has unique or infinitely many optimal solutions. Numerical simulation results also show that the proposed network is feasible and efficient. In addition, a general method for transforming non-linear programming problems into unconstrained problems is also proposed. ID="A1" Correspondence and offprint requests to: Dr Z Chen, Department of Electronic Engineering, Brunel University, Uxbridge, Middle-sex, UK