New fast normalized neural networks for pattern detection

Neural networks have shown good results for detecting a certain pattern in a given image. In this paper, fast neural networks for pattern detection are presented. Such processors are designed based on cross correlation in the frequency domain between the input image and the input weights of neural networks. This approach is developed to reduce the computation steps required by these fast neural networks for the searching process. The principle of divide and conquer strategy is applied through image decomposition. Each image is divided into small in size sub-images and then each one is tested separately by using a single fast neural processor. Furthermore, faster pattern detection is obtained by using parallel processing techniques to test the resulting sub-images at the same time using the same number of fast neural networks. In contrast to fast neural networks, the speed up ratio is increased with the size of the input image when using fast neural networks and image decomposition. Moreover, the problem of local sub-image normalization in the frequency domain is solved. The effect of image normalization on the speed up ratio of pattern detection is discussed. Simulation results show that local sub-image normalization through weight normalization is faster than sub-image normalization in the spatial domain. The overall speed up ratio of the detection process is increased as the normalization of weights is done offline.     

New faster normalized neural networks for sub-matrix detection using cross correlation in the frequency domain and matrix decomposition

Neural networks have shown good results for detecting a certain pattern in a given image. In this paper, faster neural networks for pattern detection are presented. Such processors are designed based on cross correlation in the frequency domain between the input matrix and the input weights of neural networks. This approach is developed to reduce the computation steps required by these faster neural networks for the detection process. The principle of divide and conquer strategy is applied through matrix decomposition. Each matrix is divided into smaller in size submatrices and then each one is tested separately by using a single faster neural processor. Furthermore, faster pattern detection is obtained by using parallel processing techniques to test the resulting submatrices at the same time using the same number of faster neural networks. In contrast to faster neural networks, the speed up ratio is increased with the size of the input matrix when using faster neural networks and matrix decomposition. Moreover, the problem of local submatrix normalization in the frequency domain is solved. The effect of matrix normalization on the speed up ratio of pattern detection is discussed. Simulation results show that local submatrix normalization through weight normalization is faster than submatrix normalization in the spatial domain. The overall speed up ratio of the detection process is increased as the normalization of weights is done off line.     

A novel high-speed neural model for fast pattern recognition

Neural networks have shown good results for detecting a certain pattern in a given image. In this paper, faster neural networks for pattern detection are presented. Such processors are designed based on cross-correlation in the frequency domain between the input matrix and the input weights of neural networks. This approach is developed to reduce the computation steps required by these faster neural networks for the searching process. The principle of divide and conquer strategy is applied through matrix decomposition. Each matrix is divided into smaller in size sub-matrices and then each one is tested separately using a single faster neural processor. Furthermore, faster pattern detection is obtained using parallel processing techniques to test the resulting submatrices at the same time using the same number of faster neural networks. In contrast to faster neural networks, the speed up ratio is increased with the size of the input matrix when using faster neural networks and matrix decomposition. Moreover, the problem of local sub-matrix normalization in the frequency domain is solved. The effect of matrix normalization on the speed up ratio of pattern detection is discussed. Simulation results show that local sub-matrix normalization through weight normalization is faster than sub-matrix normalization in the spatial domain. The overall speed up ratio of the detection process is increased as the normalization of weights is done off line.     

Fixed-time synchronization in multilayer networks with delay Cohen–Grossberg neural subnets via adaptive quantitative control

In this paper, fixed-time synchronization of nonlinear stochastic coupling multilayer neural networks is studied. The neural subnets in the multilayer networks are delay Cohen–Grossberg neural networks (DCGNNs). To overcome uncertain factors, we designed an adaptive delay-dependent controller in synchronization. To describe constraints of communication and other related problems in networks, which are due to limitations for bit rates and bandwidths in communication channels, an adaptive fixed-time control strategy is purposed by introducing quantization signal input. A theoretical framework about fixed-time synchronization in multilayer delay Cohen–Grossberg neural networks (MDCGNNs) is established. We find that fixed settling time is related to the scale of MDCGNNs, characteristics of the designed controller parameters, and level of quantization. Finally, the effective of the theoretical framework is validated in an example.     

Technology independent circuit sizing for standard cell based design using neural networks

This paper presents a neural network (NN) approach for modeling the time characteristics of fundamental gates of digital integrated circuits that include inverter, NAND, NOR, and XOR gates. The modeling approach presented here is technology independent, fast, and accurate, which makes it suitable for circuit simulators. Firstly transient simulations were done in order to obtain delay times for different transistor sizes and different load capacitances using AMIS 1.5 μm, TSMC 0.25 μm and TSMC 0.18 μm technology parameters with HSPICE. These delay time results constitute the inputs of NN while the outputs are transistor sizes. Then, two neural network structures, multilayer perceptron (MLP) and general regression neural network (GRNN), were compared to estimate the transistor sizes. MLP achieved 91 acceptable results through 120 test data where GRNN had 77. The important thing is that the NN is able to generalize the input–output mapping and estimates the outputs for new data which were not applied to the NN for training before. As a conclusion, fundamental gates used for standard cell based VLSI design can be sized for desired delay times using neural networks without knowing SPICE technology parameters.     

Combining seasonal time series ARIMA method and neural networks with genetic algorithms for predicting the production value of the mechanical industry in Taiwan

Supplying industrial firms with an accurate method of forecasting the production value of the mechanical industry to facilitate decision makers in precise planning is highly desirable. Numerous methods, including the autoregressive integrated-moving average (ARIMA) model and artificial neural networks can make accurate forecasts based on historical data. The seasonal ARIMA (SARIMA) model and artificial neural networks can also handle data involving trends and seasonality. Although neural networks can make predictions, deciding the most appropriate input data, network structure and learning parameters are difficult. Therefore, this article presents a hybrid forecasting method that combines the SARIMA model and neural networks with genetic algorithms. Analytical results generated by the SARIMA model are inputted as the input data of a neural network. Subsequently, the number of neurons in the hidden layer and the number of learning parameters of the neural network architecture are globally optimized using genetic algorithms. This model is subsequently adopted to forecast seasonal time series data of the production value of the mechanical industry in Taiwan. The results presented here provide a valuable reference for decision makers in industry.     

Time-delay neural networks: representation and induction offinite-state machines

In this work, we characterize and contrast the capabilities of the general class of time-delay neural networks (TDNNs) with input delay neural networks (IDNNs), the subclass of TDNNs with delays limited to the inputs. Each class of networks is capable of representing the same set of languages, those embodied by the definite memory machines (DMMs), a subclass of finite-state machines. We demonstrate the close affinity between TDNNs and DMM languages by learning a very large DMM (2048 states) using only a few training examples. Even though both architectures are capable of representing the same class of languages, they have distinguishable learning biases. Intuition suggests that general TDNNs which include delays in hidden layers should perform well, compared to IDNNs, on problems in which the output can be expressed as a function on narrow input windows which repeat in time. On the other hand, these general TDNNs should perform poorly when the input windows are wide, or there is little repetition. We confirm these hypotheses via a set of simulations and statistical analysis.     

Synchronization of Switched Neural Networks with Time‐varying Delays via Sampled‐data Control

This paper investigates sampled‐data synchronization control of switched neural networks with time‐varying delays under average dwell time. Based on the delay system method, the sampled‐data synchronization system is proposed with time‐varying delays and input delays in the unified framework for switched neural networks. By constructing a suitable Lyapunov‐Krasovskii functional and free‐weighting matrix, the relationship between the average dwell time and the maximum sampling interval is revealed to form delay‐dependent exponentially synchronization criteria. The desired mode‐dependent controller under the maximum sampling interval and decay rate is designed. Finally, two numerical examples are provided to demonstrate the effectiveness and feasibility of the proposed techniques.     

基于回归神经网络的非线性时变系统辨识

为克服基于前馈神经网络的非线性系统辨识算法存在需预先估计系统输入输出滞后阶数的缺陷，提出一种基于回归神经网络的非线性时变系统的辨识算法，针对现有的回归网络学习算法大多采用梯度算法，收敛速度缓慢问题，提出一种具有快速收敛性的扩展卡尔曼滤波学习算法，大大提高了学习收敛速度，并推导了一种基于单个神经元的局部化算法，减少了计算量，仿真实例证明，所提出的算法是有效的。     

一类具有时滞的广义Hopf ield 神经网络的全局稳定性

通过引入能量泛函，分析了一类具有时滞的广义Hopfield神经网络的全局稳定性．从理论上给出了该类网络为全局稳定的充分条件，证明了当时滞满足一个可计算的边界条件时，具有时滞的该类神经网络与相应的无时滞网络具有同样的全局稳定特性．仿真结果进一步证明了结论的有效性。     

Detection of Spikes with Artificial Neural Networks Using Raw EEG

Artificial neural networks (ANN) using raw electroencephalogram (EEG) data were developed and tested off-line to detect transient epileptiform discharges (spike and spike/wave) and EMG activity in an ongoing EEG. In the present study, a feedforward ANN with a variable number of input and hidden layer units and two output units was used to optimize the detection system. The ANN system was trained and tested with the backpropagation algorithm using a large data set of exemplars. The effects of different EEG time windows and the number of hidden layer neurons were examined using rigorous statistical tests for optimum detection sensitivity and selectivity. The best ANN configuration occurred with an input time window of 150 msec (30 input units) and six hidden layer neurons. This input interval contained information on the wave component of the epileptiform discharge which improved detection. Two-dimensional receiver operating curves were developed to define the optimum threshold parameters for best detection. Comparison with previous networks using raw EEG showed improvement in both sensitivity and selectivity. This study showed that raw EEG can be successfully used to train ANNs to detect epileptogenic discharges with a high success rate without resorting to experimenter-selected parameters which may limit the efficiency of the system.     

遗传算法优化小波神经网络的网络流量预测

为了提高网络流量的预测精度,克服小波神经网络收敛速度慢、易陷入局部最优的缺点,提出一种遗传算法优化小波神经网络的网络流量预测模型.首先计算延迟时间和嵌入维数,构建小波神经网络的学习样本,然后采用小波神经网络对网络流训练集进行学习,并采用改进遗传算法对小波神经网络参数进行全局寻优,提高收敛速度和网络学习精度,最后采用网络流量数据对模型性能进行仿真分析.结果表明,相对于对比模型,本文模型的平均误差大幅度降低,训练次数急剧减,减小了二次优化训练的次数,具有更大的实际应用价值.     

改进的选择神经网络结构的方法

有关人工神经网络作为在线状态估计器组成基于神经网络的传感器故障检测的研究已有报导。但是这些文章中没有提到如何选择神经网络的结构。神经网络的输入延迟数和隐基单元数影响其对系统的拟合精度,从而影响故障检测的灵敏度。研究了一些现有的选择神经网络结构的方法,以系统化的交叉证实法为基础,经过改进,提出了针对作为在线状态估计器的神经网络选择输入延迟数和隐层单元数的系统化的方法,并用某船在试验场中航行时平台罗经输出的一段数据作了仿真,结果证明该法可行,具有工程实用意义。     

Effect of probabilistic inputs on neural network-based electricload forecasting

This paper presents a novel method to include the uncertainties or the weather-related input variables in neural network-based electric load forecasting models. The new method consists of traditionally trained neural networks and a set of equations to calculate the mean value and confidence intervals of the forecasted load. This method was tested for daily peak load forecasts for one year by using modified data from a large power system. The tests indicate that in addition to the confidence interval, the new method provides a more accurate mean forecast than a multilayer perceptron networks alone.     

冗余度机器人运动模型的神经网络辨识及实现

为提高网络学习速度,提出了一种新的动态神经网络结构——状态延迟输入动态递归神经网络.以德国PowerCube^TM模块化机器人为研究对象,将机器人系统返回的关节位置信息和OPTOTRAK30203维运动测量系统测得的机器人末端位置信息作为神经网络的学习样本,对包含各种影响因素的机器人运动模型进行了辨识,所得结果及误差分析,说明了SDIDRNN在学习能力上的优越性.     

基于集成BP神经网络的短期负荷预测

利用混沌相空间重构理论对负荷时间序列研究,用改进的C_C方法求得时间延迟τ和嵌入维数m,得到系统最大李雅普诺夫指数,证明其具有混沌特性.对样本数据相空间重构,构建多个BP神经网络的预测子模型,所有子模型同步预测的加权平均作为集成负荷预测值.在线采集负荷数据,利用增量式训练获取新的预测子模型,按“先入先出”顺序进行BP神经网络集成更新.将预测结果同普通BP神经网络预测结果进行对比,结果证明这种方法提高了预测精度.     

基于分散神经网络的多指灵巧手位置控制

针对多指灵巧手钢缆传动系统的非线性，提出一种基于分散神经网络的位置控制方法．通过 对复杂的钢缆传动系统施加不同的输入可以得到特定的相对简单的输入输出数据，利用这种 特定的输入输出数据学习传动系统的非线性关系得到多个分散的神经网络，再根据传动系统 的结构特性用分散的神经网络求取钢缆传动系统的逆模型，用于直接逆控制，从而达到补偿 非线性误差的目的．同时应用在线神经网络的适时补偿使系统长时间保持良好的运行状态． 实验证明这种方法可大大提高位置跟踪精度，取得比较满意的结果．     

Predictive modeling for wastewater applications: Linear and nonlinear approaches

This study compares the multi-period predictive ability of linear ARIMA models to nonlinear time delay neural network models in water quality applications. Comparisons are made for a variety of artificially generated nonlinear ARIMA data sets that simulate the characteristics of wastewater process variables and watershed variables, as well as two real-world wastewater data sets. While the time delay neural network model was more accurate for the two real-world wastewater data sets, the neural networks were not always more accurate than linear ARIMA for the artificial nonlinear data sets. In some cases of the artificial nonlinear data, where multi-period predictions are made, the linear ARIMA model provides a more accurate result than the time delay neural network. This study suggests that researchers and practitioners should carefully consider the nature and intended use of water quality data if choosing between neural networks and other statistical methods for wastewater process control or watershed environmental quality management.     

基于神经网络的周期扰动非线性系统自适应渐近跟踪控制

针对一类具有周期扰动和输入时滞的不确定非线性系统,提出一种基于神经网络的自适应动态面控制方案.将径向基函数神经网络和傅里叶级数展开结合,构造一种混合函数逼近器来逼近系统中未知的周期扰动函数.通过引入一个积分项解决输入时滞问题,同时采用带有非线性滤波器的动态面控制方法,避免自适应反推控制方法中普遍存在的复杂性爆炸问题.所...     

Long future frame prediction using optical flow-informed deep neural networks for enhancement of robotic teleoperation in high latency environments

High latency in teleoperation has a significant negative impact on operator performance. While deep learning has revolutionized many domains recently, it has not previously been applied to teleoperation enhancement. We propose a novel approach to predict video frames deep into the future using neural networks informed by synthetically generated optical flow information. This can be employed in teleoperated robotic systems that rely on video feeds for operator situational awareness. We have used the image-to-image translation technique as a basis for the prediction of future frames. The Pix2Pix conditional generative adversarial network (cGAN) has been selected as a base network. Optical flow components reflecting real-time control inputs are added to the standard RGB channels of the input image. We have experimented with three data sets of 20,000 input images each that were generated using our custom-designed teleoperation simulator with a 500-ms delay added between the input and target frames. Structural Similarity Index Measures (SSIMs) of 0.60 and Multi-SSIMs of 0.68 were achieved when training the cGAN with three-channel RGB image data. With the five-channel input data (incorporating optical flow) these values improved to 0.67 and 0.74, respectively. Applying Fleiss' κ gave a score of 0.40 for three-channel RGB data, and 0.55 for five-channel optical flow-added data. We are confident the predicted synthetic frames are of sufficient quality and reliability to be presented to teleoperators as a video feed that will enhance teleoperation. To the best of our knowledge, we are the first to attempt to reduce the impacts of latency through future frame prediction using deep neural networks.