An adaptive learning algorithm for a wavelet neural network

Abstract: An optimal online learning algorithm of a wavelet neural network is proposed. The algorithm provides not only the tuning of synaptic weights in real time, but also the tuning of dilation and translation factors of daughter wavelets. The algorithm has both tracking and smoothing properties, so the wavelet networks trained with this algorithm can be efficiently used for prediction, filtering, compression and classification of various non-stationary noisy signals.     

A recurrent log-linearized Gaussian mixture network

Context in time series is one of the most useful and interesting characteristics for machine learning. In some cases, the dynamic characteristic would be the only basis for achieving a possible classification. A novel neural network, which is named "a recurrent log-linearized Gaussian mixture network (R-LLGMN)," is proposed in this paper for classification of time series. The structure of this network is based on a hidden Markov model (HMM), which has been well developed in the area of speech recognition. R-LLGMN can as well be interpreted as an extension of a probabilistic neural network using a log-linearized Gaussian mixture model, in which recurrent connections have been incorporated to make temporal information in use. Some simulation experiments are carried out to compare R-LLGMN with the traditional estimator of HMM as classifiers, and finally, pattern classification experiments for EEG signals are conducted. It is indicated from these experiments that R-LLGMN can successfully classify not only artificial data but real biological data such as EEG signals.     

The speaker identification by using genetic wavelet adaptive network based fuzzy inference system

In this paper, an intelligent speaker identification system is presented for speaker identification by using speech/voice signal. This study includes both combination of the adaptive feature extraction and classification by using optimum wavelet entropy parameter values. These optimum wavelet entropy values are obtained from measured Turkish speech/voice signal waveforms using speech experimental set. It is developed a genetic wavelet adaptive network based on fuzzy inference system (GWANFIS) model in this study. This model consists of three layers which are genetic algorithm, wavelet and adaptive network based on fuzzy inference system (ANFIS). The genetic algorithm layer is used for selecting of the feature extraction method and obtaining the optimum wavelet entropy parameter values. In this study, one of the eight different feature extraction methods is selected by using genetic algorithm. Alternative feature extraction methods are wavelet decomposition, wavelet decomposition – short time Fourier transform, wavelet decomposition – Born–Jordan time–frequency representation, wavelet decomposition – Choi–Williams time–frequency representation, wavelet decomposition – Margenau–Hill time–frequency representation, wavelet decomposition – Wigner–Ville time–frequency representation, wavelet decomposition – Page time–frequency representation, wavelet decomposition – Zhao–Atlas–Marks time–frequency representation. The wavelet layer is used for optimum feature extraction in the time–frequency domain and is composed of wavelet decomposition and wavelet entropies. The ANFIS approach is used for evaluating to fitness function of the genetic algorithm and for classification speakers. It has been evaluated the performance of the developed system by using noisy Turkish speech/voice signals. The test results showed that this system is effective in detecting real speech signals. The correct classification rate is about 91% for speaker classification.     

Probabilistic neural networks combined with wavelet coefficients for analysis of electroencephalogram signals

Abstract: In this paper, the probabilistic neural network is presented for classification of electroencephalogram (EEG) signals. Decision making is performed in two stages: feature extraction by wavelet transform and classification using the classifiers trained on the extracted features. The purpose is to determine an optimum classification scheme for this problem and also to infer clues about the extracted features. The present research demonstrates that the wavelet coefficients obtained by the wavelet transform are features which represent the EEG signals well. The conclusions indicate that the probabilistic neural network trained on the wavelet coefficients achieves high classification accuracies (the total classification accuracy is 97.63%).     

A wavelet neural network for the detection of heart valve diseases

An expert system is presented for interpretation of the Doppler signals of heart valve diseases based on pattern recognition. We deal in particular with the combination of feature extraction and classification from measured Doppler signal waveforms at the heart valve using Doppler ultrasound. A wavelet neural network model developed by us is used. The model consists of two layers: a wavelet layer and a multilayer perceptron. The wavelet layer used for adaptive feature extraction in the time–frequency domain is composed of wavelet decomposition and wavelet entropy. The multilayer perceptron used for classification is a feedforward neural network. The performance of the developed system has been evaluated in 215 samples. The test results show that this system is effective to detect Doppler heart sounds. The classification rate averaged 91% correct for 123 test subjects.     

An expert system for speaker identification using adaptive wavelet sure entropy

In this study, an expert speaker identification system is presented for speaker identification using Turkish speech signals. Here, a discrete wavelet adaptive network based fuzzy inference system (DWANFIS) model is used for this aim. This model consists of two layers: discrete wavelet and adaptive network based fuzzy inference system. The discrete wavelet layer is used for adaptive feature extraction in the time–frequency domain and is composed of discrete wavelet decomposition and discrete wavelet entropy. The performance of the used system is evaluated by using repeated speech signals. These test results show the effectiveness of the developed intelligent system presented in this paper. The rate of correct classification is about 90.55% for the sample speakers.     

A novel approach to wavelet selection and tree kernel construction for diagnosis of rolling element bearing fault

A novel methodology for early diagnosis of rolling element bearing fault is employed based on continuous wavelet transform (CWT) and support vector machine (SVM). CWT is especially suited for analyzing non-stationary signals in time–frequency domain where time information is retained as well as frequency content. To better approximate non-stationary vibration signals from rolling element bearing, a wavelet choice criterion is established to select an appropriate mother wavelet for feature extraction. The Shannon wavelet is picked out of several considered wavelets. The classification tree kernels (CTK) are constructed to address nonlinear classification of the characteristic samples derived from the wavelet coefficients. By using Fuzzy pruning strategy, a large variety of classification trees are generated. The trees with diverse structures can effectively explore intrinsic information among samples. Then, the tree kernel matrices can be acquired through ensemble statistical learning, which eventually reveal the similarity of samples objectively and stably. Under such architecture of kernel methods, a classification tree kernel based support vector machine (CTKSVM) is proposed to identify bearing fault. The performance of the methodology involving CWT and CTKSVM (CWT–CTKSVM) is evaluated by cross validation and independent test. The results show that the CWT–CTKSVM totally is superior to other SVM methods with common kernels. Therefore, it is a prospective technique for detection and identification of rolling element bearing fault.     

RETRACTED ARTICLE: Nonlinear dynamic systems identification based on dynamic wavelet neural units

In this paper, a dynamic wavelet network (DWN) is proposed and applied to identify black box models of the process. The well-known delta-rule is extended to the dynamic delta-rule in order to optimize wavelet network parameters. A chemical process was chosen as a realistic nonlinear system to demonstrate the identification performance. A comparison was made between the approach presented in this paper and dynamic multi layer perceptron neural networks.     

基于小波系数聚类的特征提取分类方法

神经网络是一种普遍采用的模式分类方法,当对样本的抽样数目较大时,神经网络结构复杂,训练时间激增,分类性能下降,针对这一问题,提出一种基于快速小波变换特征提取的分类方法。着先对婆婆以系数矩阵的每行进行聚类,表达重要频率范围内小波系数矩阵的行有较多的聚类数,从而大大减少了神经网络的输入数,而同时保留了有用的信息。特征提取后,采用小波系数的能量值特征量,应用径向基函数网络识别肺发出的各种不同的声音,实验证明：该方法有较高的识别率。     

基于自适应混合优化的电力数据预测方法

电力数据易受气候、季节、节假日等因素影响,出现不同波动特征.针对不同特征电力数据预测精度不高、预测方法泛化能力弱等问题,提出基于自适应混合优化的电力数据预测方法 .通过使用小波变换和平稳性分析,将电力数据自适应地分解为包含趋势、季节和周期信息的非平稳序列和多个平稳序列;使用状态转移算法分别优化长短时记忆深度学习网络和自回归移动平均模型,对非平稳序列和平稳序列分别拟合、预测;对预测的各序列进行重构,得到最终预测结果.在电力系统数据上进行多步预测,对比实验表明:与其他方法相比,所提方法不仅具有更高的预测精度,还具有较强的泛化能力.     

An improved convolutional neural network with an adaptable learning rate towards multi-signal fault diagnosis of hydraulic piston pump

Hydraulic piston pump is a vital component of hydraulic transmission system and plays a critical role in some modern industrials. On account of the deficiencies of traditional fault diagnosis in preprocessing of original data and feature extraction, the intelligent methods based on deep learning accomplish the automatic learning of fault information by integrating feature extraction and classification. As a popular deep learning model, convolutional neural network (CNN) has been demonstrated to be potent and effective in image classification. In this research, an improved intelligent method based on CNN with adapting learning rate is constructed for fault diagnosis of a hydraulic piston pump. Firstly, three raw signals are converted into two dimensional time–frequency images by continuous wavelet transform, including vibration signal, pressure signal and sound signal. Secondly, an improved deep CNN model is built with an adaptive learning rate strategy for identifying the different fault types. Moreover, t-distributed stochastic neighbor embedding is employed to visualize the distribution of features learned by the main layers of CNN model. Confusion matrix is used to analyze the classification accuracy of each fault type. Compared with the CNN model without adapting learning rate, the improved model achieves a higher accuracy based on the selected three kinds of signals. Experiments indicate that the improved CNN model can effectively and accurately identify various faults for a hydraulic piston pump.     

A self-adaptive data analysis for fault diagnosis of an automotive air-conditioner blower

This paper presents a fault diagnosis system for an automotive air-conditioner blower based on a noise emission signal using a self-adaptive data analysis technique. The proposed diagnosis system consists of feature extraction using the empirical mode decomposition (EMD) method and fault classification using the artificial neural network technique. The EMD method has been developed quite recently to adaptively decompose the non-stationary and non-linear signals. It sifts the complex signal of time series without losing its original properties and then obtains some useful intrinsic mode function (IMF) components. Calculating the energy of each component can reduce the computation dimensions and enhance classification performance. These energy features of various fault conditions are used as inputs to train the artificial neural network. In the fault classification, the probabilistic neural network (PNN) is used to verify the performance of the proposed system and compare with the traditional technique, back-propagation neural network (BPNN). The experimental results indicated the proposed technique performed well for quickly and accurately estimating fault conditions.     

Adaptive wavelet network for multiple cardiac arrhythmias recognition

This paper proposes a method for electrocardiogram (ECG) heartbeat detection and recognition using adaptive wavelet network (AWN). The ECG beat recognition can be divided into a sequence of stages, starting with feature extraction from QRS complexes, and then according to characteristic features to identify the cardiac arrhythmias including the supraventricular ectopic beat, bundle branch ectopic beat, and ventricular ectopic beat. The method of ECG beats is a two-subnetwork architecture, Morlet wavelets are used to enhance the features from each heartbeat, and probabilistic neural network (PNN) performs the recognition tasks. The AWN method is used for application in a dynamic environment, with add-in and delete-off features using automatic target adjustment and parameter tuning. The experimental results used from the MIT-BIH arrhythmia database demonstrate the efficiency of the proposed non-invasive method. Compared with conventional multi-layer neural networks, the test results also show accurate discrimination, fast learning, good adaptability, and faster processing time for detection.     

基于小波神经网络的光谱数据压缩与分类研究

文中介绍了一种基于小波分析而构造的神经网络模型－小波神经网络,利用它并适当选取网络结构和小波基。实现了对化学物质红外光谱数据的压缩表达和分类,实验表明,网络在大幅度压缩数据的同时能很好地恢复原始光谱,较准确地反映吸收峰的和强度。在分类方面它比其它网络具有更高的分辨率和特征提取能力。     

On-Line Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms

Outlier detection is a fundamental issue in data mining, specifically in fraud detection, network intrusion detection, network monitoring, etc. SmartSifter is an outlier detection engine addressing this problem from the viewpoint of statistical learning theory. This paper provides a theoretical basis for SmartSifter and empirically demonstrates its effectiveness. SmartSifter detects outliers in an on-line process through the on-line unsupervised learning of a probabilistic model (using a finite mixture model) of the information source. Each time a datum is input SmartSifter employs an on-line discounting learning algorithm to learn the probabilistic model. A score is given to the datum based on the learned model with a high score indicating a high possibility of being a statistical outlier. The novel features of SmartSifter are: (1) it is adaptive to non-stationary sources of data; (2) a score has a clear statistical/information-theoretic meaning; (3) it is computationally inexpensive; and (4) it can handle both categorical and continuous variables. An experimental application to network intrusion detection shows that SmartSifter was able to identify data with high scores that corresponded to attacks, with low computational costs. Further experimental application has identified a number of meaningful rare cases in actual health insurance pathology data from Australia's Health Insurance Commission.     

基于改进粒子群优化算法的新型小波神经网研究

本文提出了改进的粒子群优化算法（Improved Particle Swarm Optimization,IPSO）的新型BP 小波 神经网络,并且对非线性辨识问题进行了仿真实验．实验结果表明,基于改进的粒子群优化算法的BP 小波网 络不仅具有小波分析良好的局部特性以及神经网络的学习、分类能力,而且具有粒子群优化算法全局快速寻 优的特点．与简单的粒子群优化算法相比,该方法在收敛性和稳定性方面都有了较明显的提高,验证了它的 合理性和有效性．     

基于神经网络观测器的船舶轨迹跟踪递归滑模 动态面输出反馈控制

针对三自由度全驱动船舶速度向量不可测问题,考虑船舶模型参数和外部环境扰动均未知的情况,提出一种基于神经网络观测器的船舶轨迹跟踪递归滑模动态面输出反馈控制方法.该方法设计神经网络自适应观测器估计船舶速度向量,且利用神经网络逼近模型参数不确定项,综合考虑船舶位置和速度误差之间关系构造递归滑模面,再采用动态面控制技术设计轨迹跟踪控制律和参数自适应律,并引入低频增益学习方法消除外界扰动导致的高频振荡控制信号.选取李雅普诺夫函数证明了该控制律能够保证轨迹跟踪闭环系统内所有信号的一致最终有界性.最后,基于一艘供给船进行仿真验证,结果表明,船舶轨迹跟踪响应速度快,所设计控制器对系统模型参数摄动及外界扰动具有较强的鲁棒性.     

交通流量小波神经网络多步预测研究

针对交通流量混沌时间序列多步预测的问题,提出了一种基于混沌机理的小波神经网络(WNN)快速学习算法.通过将混沌理论和小波分析相结合,建立了交通流量时间序列WNN模型;阐述了混沌学习算法的机理,设计了交通流量WNN混沌时间序列自适应学习算法.仿真试验结果表明,该算法的多步预测性能明显优于应用BP网络和非混沌算法的小波神经网络.     

基于深度度量学习的电机故障诊断

深度学习以其强大的自适应特征提取和分类能力在机械大数据处理方面取得了丰硕的成果，由于电机结构的复杂性，其信号表现出的非平稳、非线性和复杂多样等特点，使得传统分类方法中的Softmax分类器+交叉熵损失函数对电机故障诊断力不从心。根据电机信号非平稳、数据量大等特点，结合短时傅里叶变换(STFT)与深度学习中的卷积神经网络(CNN)算法和Triplet Loss三元组思想，提出了深度度量学习电机故障诊断方法。该方法能将电机故障信号转换成时频谱图，同时构建CNN，将预处理后的样本用于CNN的训练，采用Triplet Loss作为损失函数度量故障数据高维特征间的距离，并结合标签有监督地微调整个网络，从而实现准确的电机故障诊断。实验表明该方法在处理复杂数据时能够度量特征在高维空间中的距离，高效完成故障诊断任务，弥补了交叉熵函数的不足。