前馈神经网络凸优化算法及其仿真应用

该文利用凸函数共轭性质中的Young不等式构造前馈神经网络优化目标函数。这个优化目标函数若固定权值,对隐层输出来说为凸函数;若固定隐层输出,对权值来说为凸函数。因此,此目标函数不存在局部最小。此目标函数的优化速度快,大大提高了前馈神经网络的学习效率。仿真试验表明,与传统算法如误差反向传播算法或BP算法和含势态因子(Momentum factor)的BP算法及现有的分层优化算法相比,新算法能加快收敛速度,并降低学习误差。利用这种快速算法对矿体进行仿真预测,取得了良好效果。     

联合自注意力和循环网络的图像标题生成

目前大多数图像标题生成模型都是由一个基于卷积神经网络(Convolutional Neural Network,CNN)的图像编码器和一个基于循环神经网络(Recurrent Neural Network,RNN)的标题解码器组成.其中图像编码器用于提取图像的视觉特征,标题解码器基于视觉特征通过注意力机制来生成标题.然...     

Automated Recurrent Neural Network Design to Model the Dynamics of Complex Systems

A general purpose implementation of the tabu search metaheuristic, called Universal Tabu Search, is used to optimally design a locally recurrent neural network architecture. The design of a neural network is a tedious and time consuming trial and error operation that leads to structures whose optimality is not guaranteed. In this paper, the problem of choosing the number of hidden neurons and the number of taps and delays in the FIR and IIR network synapses is formalised as an optimisation problem, whose cost function to be minimised is the network error calculated on a validation data set. The performance of the proposed approach has been tested on the problem of modelling the dynamics of a non-isothermal, continuously stirred tank reactor, in two different operating conditions: when a first order exothermic reaction is occurring; and when two consecutive first order reactions lead to a chaotic behaviour. Comparisons with alternative neural approaches are reported, showing the usefulness of the proposed method.     

Application of a Recurrent Neural Network to Space Deversity in SDMA and CDMA Mobile Communication Systems

Linear and non-linear adaptive algorithms are investigated for Space Division Multiple Access (SDMA). SDMA is one of the emerging techniques for multiple access of users in mobile radio, which uses spatial distribution of users for their differentiation. The performance of the linear Square Root Kalman (SRK) algorithm for SDMA is compared to that of the non-linear Recurrent Neural Network (RNN) technique. The proposed SDMA-RNN technique is evaluated over Rician fading channels, and it shows improved Bit Error Rate (BER) performance in comparison with the linear SRK-based technique. The performance of SDMA-RNN is also compared with that of Code Division Multiple Access (CDMA) systems, showing that it could be used as a viable alternative scheme for multiple access of users. Finally, a Hybrid CDMA-SDMA system is proposed combining CDMA and SDMA-RNN systems. Hybrid CDMA-SDMA exhibits a very good potential for increase in the capacity and the performance of mobile communications systems.     

On Postprocessing of Neural Network Prediction of Polyproline Type II Secondary Structures: Network Spectrum, Response Analysis, and Scattering

In our previous studies, we predicted protein secondary structures of polyproline type II by applying feedforward perceptron neural networks with the backpropagation learning algorithm. With a uniformly distributed test set, prediction succeeded in approximately 74% of cases, which is indeed a highvalue for a prediction problem in bioinformatics. To enable the deeper investigation of the problem of incorrect classifications and the prediction as whole, we developed new techniques for the analysis of learning data and for the decision making of neural networks with the polyproline type II material. We briefly present the results of a neural network in this context, and the techniques developed for postprocessing. The spectrum of a neural network was used for a sophisticated frequency and response analysis to describe the interactions of different amino acids in connection with the windowing employed in the preprocessing of protein data. Scattering by means of Hamming distances was used to search for local clusters and to describe learnability of data in the pattern space.