Stochastic processes in machine intelligence: Neural structures based on the model of the quantum harmonic oscillator

This paper studies neural structures with weights that follow the model of the quantum harmonic oscillator. The proposed neural networks have stochastic weights which are calculated from the solution of Schrödinger’s equation under the assumption of a parabolic (harmonic) potential. These weights correspond to diffusing particles, which interact with each other as the theory of Brownian motion (Wiener process) predicts. It is shown that conventional neural networks and learning algorithms based on error gradient can be conceived as a subset of the proposed quantum neural structures. The learning of the stochastic weights (convergence of the diffusing particles to an equilibrium) is analyzed. In the case of associative memories the proposed neural model results in an exponential increase of patterns storage capacity (number of attractors).     

Classical and superposed learning for quantum weightless neural networks

A supervised learning algorithm for quantum neural networks (QNN) based on a novel quantum neuron node implemented as a very simple quantum circuit is proposed and investigated. In contrast to the QNN published in the literature, the proposed model can perform both quantum learning and simulate the classical models. This is partly due to the neural model used elsewhere which has weights and non-linear activations functions. Here a quantum weightless neural network model is proposed as a quantisation of the classical weightless neural networks (WNN). The theoretical and practical results on WNN can be inherited by these quantum weightless neural networks (qWNN). In the quantum learning algorithm proposed here patterns of the training set are presented concurrently in superposition. This superposition-based learning algorithm (SLA) has computational cost polynomial on the number of patterns in the training set.     

扩散遗传算法及其应用研究

针对遗传算法在局部搜索能力方面的缺陷,提出了一种基于扩散算子的遗产算法(简称扩散遗产算法)。该算法中包含的扩散算子是变异算子,其主要作用是在遗传搜索中进行局部搜索。用扩散遗传算法和实数编码遗传算法分别训练用于解XOR问题的神经网络,对比结果表明,论文提出的算法兼具强的全局搜索能力和局部搜索能力,因此,该算法可以不借助其它局部搜索算法而单独作为神经网络训练算法,从而简化训练算法,提高训练效率。该算法对提高遗传算法搜索效率和求解精度具有重要的意义。     

一种量子衍生神经网络模型

为提高神经网络的逼近能力,通过在普通BP网络中引入量子旋转门,提出了一种新颖的量子衍生神经网络模型. 该模型隐层由量子神经元组成,每个量子神经元携带一组量子旋转门,用于更新隐层的量子权值,输入层和输出层均为普通神经元. 基于误差反传播算法设计了该模型的学习算法. 模式识别和函数逼近的实验结果验证了提出模型及算法的有效性.     

Functional abilities of a stochastic logic neural network

The authors have studied the information processing ability of stochastic logic neural networks, which constitute one of the pulse-coded artificial neural network families. These networks realize pseudoanalog performance with local learning rules using digital circuits, and therefore suit silicon technology. The synaptic weights and the outputs of neurons in stochastic logic are represented by stochastic pulse sequences. The limited range of the synaptic weights reduces the coding noise and suppresses the degradation of memory storage capacity. To study the effect of the coding noise on an optimization problem, the authors simulate a probabilistic Hopfield model (Gaussian machine) which has a continuous neuron output function and probabilistic behavior. A proper choice of the coding noise amplitude and scheduling improves the network's solutions of the traveling salesman problem (TSP). These results suggest that stochastic logic may be useful for implementing probabilistic dynamics as well as deterministic dynamics.     

量子神经网络在电力谐波检测中的应用

该文利用复数BP学习算法,构造出量子神经元模型[1],并结合神经网络技术与量子理论,生成更有效的泛化和学习能力的量子神经网络。基于三层量子神经网络实现对谐波参数的检测,并以3次谐波和5次谐波为例,描述了该网络的训练流程和训练样本的构成。量子神经网络的实现采用Matlab进行编程,首先利用训练样本训练量子网络,之后检测构造的未训练样本数据集,通过仿真结果验证了该方法的可行性。该方法在谐波检测中具有较高的灵活性和精度,且对采样数目没有严格的限制,训练好的量子神经网络模型可用于谐波源固定的场合。     

一种新型随机二进制神经网络

二进制数据表示具有简洁高效的特点,随机噪声有助于系统摆脱局部极小.新型的随 机神经网络模型采用随机加权联接,内部数据表示为随机二进制序列形式,实现十分高效.文中 分别就前馈型网络和反馈型网络进行了深入的讨论,给出了前馈型网络的梯度下降学习算法, 为反馈型网络设计了快速有效的模拟退火算法和渐进式Boltzmann学习算法.通过对PARITY 问题的测试,发现了新模型的一些有趣特征,实验结果表明梯度下降学习效果显著.利用渐进式 Boltzmann学习,反馈型网络被成功地用于带噪声人脸识别.     

Cooperative behavior of nano-robots as an analogous of the quantum harmonic oscillator

A multi-robot system that consists of N nano-robots is studied. It is assumed that the robots correspond to diffusing particles, and interact to each other as the theory of Brownian motion predicts. Brownian motion is the analogous of the quantum harmonic oscillator (Q.H.O.), i.e., of Schrödinger’s equation under harmonic (parabolic) potential. It is shown that the motion of the robots can be described by Langevin’s equation which is a stochastic linear differential equation. It is proved that Langevin’s equation is a generalization of conventional gradient algorithms. Therefore the kinematic models of mobile robots which follow conventional gradient algorithms can be considered as a subcase of the kinematic models which are derived from the diffusion analogous of the Q.H.O model.     

Finding Minimum Vertex Covering in Stochastic Graphs: A Learning Automata Approach

Structural and behavioral parameters of many real networks such as social networks are unpredictable, uncertain, and have time-varying parameters, and for these reasons, deterministic graphs for modeling such networks are too restrictive to solve most of the real-network problems. It seems that stochastic graphs, in which weights associated to the vertices are random variables, might be better graph models for real-world networks. Once we use a stochastic graph as the model for a network, every feature of the graph such as path, spanning tree, clique, dominating set, and cover set should be treated as a stochastic feature. For example, choosing a stochastic graph as a graph model of an online social network and defining community structure in terms of clique, the concept of a stochastic clique may be used to study community structures’ properties or define spreading of influence according to the coverage of influential users; the concept of stochastic vertex covering may be used to study spread of influence. In this article, minimum vertex covering in stochastic graphs is first defined, and then four learning, automata-based algorithms are proposed for solving a minimum vertex-covering problem in stochastic graphs where the probability distribution functions of the weights associated with the vertices of the graph are unknown. It is shown that through a proper choice of the parameters of the proposed algorithms, one can make the probability of finding minimum vertex cover in a stochastic graph as close to unity as possible. Experimental results on synthetic stochastic graphs reveal that at a certain confidence level the proposed algorithms significantly outperform the standard sampling method in terms of the number of samples needed to be taken from the vertices of the stochastic graph.     

Learning continuous trajectories in recurrent neural networks withtime-dependent weights

The paper is concerned with a general learning (with arbitrary criterion and state-dependent constraints) of continuous trajectories by means of recurrent neural networks with time-varying weights. The learning process is transformed into an optimal control framework, where the weights to be found are treated as controls. A learning algorithm based on a variational formulation of Pontryagin's maximum principle is proposed. This algorithm is shown to converge, under reasonable conditions, to an optimal solution. The neural networks with time-dependent weights make it possible to efficiently find an admissible solution (i.e., initial weights satisfying state constraints) which then serves as an initial guess to carry out a proper minimization of a given criterion. The proposed methodology may be directly applicable to both classification of temporal sequences and to optimal tracking of nonlinear dynamic systems. Numerical examples are also given which demonstrate the efficiency of the approach presented.     

量子–经典混合神经网络及其故障诊断应用

量子神经网络由于结合了量子计算和神经网络的优点, 近年来受到了广泛的关注. 然而由于目前量子计算 资源受限(如量子比特数、量子逻辑门的保真度等)以及贫瘠高原现象(量子神经网络优化过程中解空间变得平坦时 出现的训练困难)的存在, 量子神经网络当前还难以大规模训练. 针对上述问题, 本文面向量子–经典混合神经网络 模型提出了一种基于无监督学习的特征提取方法. 所采用的无监督学习方法结合了量子自编码器和K-medoids聚类 方法, 可用于多层次结构的特征学习. 该方法创新地利用了K-mediods方法对训练得到的量子自编码器进行聚类, 以 最大化量子自编码器性质的差异. 进一步, 本文在轴承异常检测问题上, 通过实验验证了所提出的无监督特征提取 方法的有效性和实用性, 测试集准确率在二分类、四分类和十分类分别达到100%, 89.6%和81.6%.     

Evolutionary neural networks for anomaly detection based on the behavior of a program.

The process of learning the behavior of a given program by using machine-learning techniques (based on system-call audit data) is effective to detect intrusions. Rule learning, neural networks, statistics, and hidden Markov models (HMMs) are some of the kinds of representative methods for intrusion detection. Among them, neural networks are known for good performance in learning system-call sequences. In order to apply this knowledge to real-world problems successfully, it is important to determine the structures and weights of these call sequences. However, finding the appropriate structures requires very long time periods because there are no suitable analytical solutions. In this paper, a novel intrusion-detection technique based on evolutionary neural networks (ENNs) is proposed. One advantage of using ENNs is that it takes less time to obtain superior neural networks than when using conventional approaches. This is because they discover the structures and weights of the neural networks simultaneously. Experimental results with the 1999 Defense Advanced Research Projects Agency (DARPA) Intrusion Detection Evaluation (IDEVAL) data confirm that ENNs are promising tools for intrusion detection.     

用遗传算法优化Boltzmann机

Boltzmann机是一种应用广泛的随机神经网络。它通过模拟退火算法进行网络学习,能取得一个全局或接近全局最优的最优值;通过期望网络模式和实际学习得到的网络模式比较来调节网络的权值,使网络能尽可能地达到或逼近期望的网络模式。将遗传算法运用到Boltzmann机的网络学习中,在对BM机编码后,通过选择、交叉和变异等遗传操作算子对网络进行训练,调整网络的权值,使适应度函数值大的网络保留下来,最终使网络达到期望的模式。通过实例验证,这是一种简单可行的调节网络权值的方法。     

基于量子神经网络的油田水淹层识别方法

针对油藏测井解释中的水淹层识别问题,提出一种量子神经网络模型。该模型用量子旋转门更新量子比特的相位,用受控旋转门实现网络的非线性映射功能。网络可调参数为量子旋转门的旋转角度和受控非门的控制参数。基于梯度下降法设计了学习算法。仿真结果表明,该模型的预测能力优于普通BP网络、模糊神经网络和过程神经网络等其他方法。     

多层随机神经网络em算法

本文讨论了基于微分流形框架随机神经网络学习算法，称为ｅｍ学习算法；对于多层随机神经网络模型，我们从微分流形的角度分析它的对偶平坦流形结构，描述ｅｍ算法对于多层前馈随机神经网络模型学习算法实现和加速技术。     

Prediction and identification using wavelet-based recurrent fuzzy neural networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.     

A rapid learning and dynamic stepwise updating algorithm for flatneural networks and the application to time-series prediction

A fast learning algorithm is proposed to find an optimal weights of the flat neural networks (especially, the functional-link network). Although the flat networks are used for nonlinear function approximation, they can be formulated as linear systems. Thus, the weights of the networks can be solved easily using a linear least-square method. This formulation makes it easier to update the weights instantly for both a new added pattern and a new added enhancement node. A dynamic stepwise updating algorithm is proposed to update the weights of the system on-the-fly. The model is tested on several time-series data including an infrared laser data set, a chaotic time-series, a monthly flour price data set, and a nonlinear system identification problem. The simulation results are compared to existing models in which more complex architectures and more costly training are needed. The results indicate that the proposed model is very attractive to real-time processes.     

基于受控旋转门的量子神经网络模型算法及应用

提出一种量子神经网络模型及算法．首先借鉴受控非门的含义提出一种受控量子旋转门,基于该门的物理意义,提出一种量子神经元模型,该模型包含对输入量子比特相位的旋转角度和对旋转角度的控制量两种设计参数;然后基于上述量子神经元提出一种量子神经网络模型,基于梯度下降法详细设计了该模型的学习算法：最后通过模式识别和时间序列预测两个仿...     

ANASA-a stochastic reinforcement algorithm for real-valued neuralcomputation

This paper introduces ANASA (adaptive neural algorithm of stochastic activation), a new, efficient, reinforcement learning algorithm for training neural units and networks with continuous output. The proposed method employs concepts, found in self-organizing neural networks theory and in reinforcement estimator learning algorithms, to extract and exploit information relative to previous input pattern presentations. In addition, it uses an adaptive learning rate function and a self-adjusting stochastic activation to accelerate the learning process. A form of optimal performance of the ANASA algorithm is proved (under a set of assumptions) via strong convergence theorems and concepts. Experimentally, the new algorithm yields results, which are superior compared to existing associative reinforcement learning methods in terms of accuracy and convergence rates. The rapid convergence rate of ANASA is demonstrated in a simple learning task, when it is used as a single neural unit, and in mathematical function modeling problems, when it is used to train various multilayered neural networks.     

非线性伺服电动机的神经网络逆控制

伺服电动机由于存在接触过程的非线性、温漂等非线性因素的影响,很难建立其精确的数学模型,使得基于数学模型的控制困难.针对伺服电动机存在的非线性问题,提出了一种新颖的基于BP神经网络直接逆控制方法.首先,利用BP神经网络建立系统的正向模型(NNI),然后,设计基于神经网络的直接逆控制器(NNC),实现了对伺服电动机的自适应控制.在Lyapunov稳定性分析的基础上,给出了BP算法学习算子的选择方案,保证神经网络权值训练的快速收敛,同时,对训练BP神经网络控制器的专用算法(specialized learning)进行改进,利用NNI的输出求取权值调整的灵敏度函数.数字仿真结果表明提出的控制算法是简单有效的.