Weighted centroid neural network for edge preserving imagecompression

An edge preserving image compression algorithm based on an unsupervised competitive neural network is proposed. The proposed neural network, the called weighted centroid neural network (WCNN), utilizes the characteristics of image blocks from edge areas. The mean/residual vector quantization (M/RVQ) scheme is utilized in this proposed approach as the framework of the proposed algorithm. The edge strength of image block data is utilized as a tool to allocate the proper code vectors in the proposed WCNN. The WCNN successfully allocates more code vectors to the image block data from edge area while it allocates less code vectors to the image black data from shade or non-edge area when compared to conventional neural networks based on VQ algorithm. As a result, a simple application of WCNN to an image compression problem gives improved edge characteristics in reconstructed images over conventional neural network based on VQ algorithms such as self-organizing map (SOM) and adaptive SOM.     

Error Concealment Using Adaptive Multilayer Perceptrons (MLPs) for Block-Based Image Coding

Image coding algorithms such as Vector Quantisation (VQ), JPEG and MPEG have been widely used for encoding image and video. These compression systems utilise block-based coding techniques to achieve a higher compression ratio. However, a cell loss or a random bit error during network transmission will permeate into the whole block, and then generate several damaged blocks. Therefore, an efficient Error Concealment (EC) scheme is essential for diminishing the impact of damaged blocks in a compressed image. In this paper, a novel adaptive EC algorithm is proposed to conceal the error for block-based image coding systems by using neural network techniques in the spatial domain. In the proposed algorithm, only the intra-frame information is used for reconstructing the image with damaged blocks. The information of pixels surrounding a damaged block is used to recover the errors using the neural network models. Computer simulation results show that the visual quality and the PSNR evaluation of a reconstructed image are significantly improved using the proposed EC algorithm.     

基于神经二部分裂结构的多智能体WSNs数据融合算法?

为进一步提高无线传感器网络WSNs( Wireless Sensor Networks)使用寿命,从提高算法数据融合效率角度出发,提出一种神经二部分裂结构的多智能体WSNs数据融合算法。首先,根据神经结构的主干与枝干承载信息量不同的原理,选取主干、枝干通讯链路并赋予较大能量,并给出主、辅中心节点选取方法;其次,设计了基于LMS的自适应加权融合算法,分别针对节点层级、枝干中心层级和主干中心层级进行逐层处理,实现了对神经二部分裂结构的数据融合;最后,通过与两种已有算法进行仿真对比,显示本文算法在Sink节点接收数据包,能耗等指标上均具有优势,验证了算法有效性。     

气体传感器阵列常用模式识别算法

模式识别在气体传感器阵列的测量中占有举足轻重的地位。介绍了k近邻法、聚类分析、判别函数分析、反向传播人工神经网络、主元分析法、概率神经网、学习向量量化、自组织映射、自适应共振网、遗传算法等气体传感器阵列常用模式识别算法的原理和特点。同时,指出了在应用中模式识别算法选择和评价的标准。     

Towards the Optimal Learning Rate for Backpropagation

A backpropagation learning algorithm for feedforward neural networks withan adaptive learning rate is derived. The algorithm is based uponminimising the instantaneous output error and does not include anysimplifications encountered in the corresponding Least Mean Square (LMS)algorithms for linear adaptive filters. The backpropagation algorithmwith an adaptive learning rate, which is derived based upon the Taylorseries expansion of the instantaneous output error, is shown to exhibitbehaviour similar to that of the Normalised LMS (NLMS) algorithm. Indeed,the derived optimal adaptive learning rate of a neural network trainedby backpropagation degenerates to the learning rate of the NLMS for a linear activation function of a neuron. By continuity, the optimal adaptive learning rate for neural networks imposes additional stabilisationeffects to the traditional backpropagation learning algorithm.     

Adaptive deadlock- and livelock-free routing in the hypercubenetwork

This paper consists of two parts. In the first one, two new algorithms for wormhole routing on the hypercube network are presented. These techniques are adaptive and are ensured to be deadlock- and livelock-free. These properties are guaranteed by using a small number of resources in the routing node. The first algorithm is adaptive and nonminimal and will be referred to as Nonminimal. In this technique, some moderate derouting is allowed in order to alleviate the potential congestion arising from highly structured communication patterns. The second algorithm, dubbed Subcubes, is adaptive and minimal, and is based on partitioning the hypercube into subcubes of smaller dimension; This technique requires only two virtual channels per physical link of the node. In the second part of the paper, a wide variety of techniques for wormhole routing in the hypercube are evaluated from an algorithmic point of view. Five partially adaptive algorithms are considered: the Hanging algorithm, the Zenith algorithm, the Hanging-Order algorithm, the Nonminimal algorithm; and the Subcubes algorithm. One oblivious algorithm, the Dimension-Order, or E-Cube routing algorithm, is also used. Finally, a Fully Adaptive Minimal algorithm is tried. A simple node model was designed and adapted to all the algorithms     

Discrete vector quantization for arbitrary distance functionestimation

There are currently many vastly different areas of research involving adaptive learning. Among them are the two areas that concern neural networks and learning automata. This paper develops a method by which the general philosophies of vector quantization (VQ) and discretized automata learning can be incorporated for the computation of arbitrary distance functions. The latter is a problem which has important applications in logistics and location analysis. The input to our problem is the set of coordinates of a large number of nodes whose internode arbitrary "distances" have to be estimated. To render the problem interesting, nontrivial, and realistic, we assume that the explicit form of this distance function is both unknown and uncomputable. Unlike traditional operations research methods, which use optimized parametric functional estimators, we have utilized discretized VQ principles to first adaptively polarize the nodes into subregions. Subsequently, the parameters characterizing the subregions are learned by using a variety of methods (including, for academic purposes, a VQ strategy in the meta-domain). After an initial training phase, a system which achieves distance estimation attempts to yield an estimate of any node-pair distance without actually deriving an explicit form for the unknown function. The algorithms have been rigorously tested for the actual road-travel distances involving cities in Turkey and the results obtained are conclusive. Indeed, these present results are the best currently available from any single or hybrid strategy.     

Online identification of nonlinear multivariable processes using self‐generating RBF neural networks

This paper addresses the problem of online model identification for multivariable processes with nonlinear and time‐varying dynamic characteristics. For this purpose, two online multivariable identification approaches with self‐organizing neural network model structures will be presented. The two adaptive radial basis function (RBF) neural networks are called as the growing and pruning radial basis function (GAP‐RBF) and minimal resource allocation network (MRAN). The resulting identification algorithms start without a predefined model structure and the dynamic model is generated autonomously using the sequential input‐output data pairs in real‐time applications. The extended Kalman filter (EKF) learning algorithm has been extended for both of the adaptive RBF‐based neural network approaches to estimate the free parameters of the identified multivariable model. The unscented Kalman filter (UKF) has been proposed as an alternative learning algorithm to enhance the accuracy and robustness of nonlinear multivariable processes in both the GAP‐RBF and MRAN based approaches. In addition, this paper intends to study comparatively the general applicability of the particle filter (PF)‐based approaches for the case of non‐Gaussian noisy environments. For this purpose, the Unscented Particle Filter (UPF) is employed to be used as alternative to the EKF and UKF for online parameter estimation of self‐generating RBF neural networks. The performance of the proposed online identification approaches is evaluated on a highly nonlinear time‐varying multivariable non‐isothermal continuous stirred tank reactor (CSTR) benchmark problem. Simulation results demonstrate the good performances of all identification approaches, especially the GAP‐RBF approach incorporated with the UKF and UPF learning algorithms. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

On Adaptive Learning Rate That Guarantees Convergence in Feedforward Networks

This paper investigates new learning algorithms (LF I and LF II) based on Lyapunov function for the training of feedforward neural networks. It is observed that such algorithms have interesting parallel with the popular backpropagation (BP) algorithm where the fixed learning rate is replaced by an adaptive learning rate computed using convergence theorem based on Lyapunov stability theory. LF II, a modified version of LF I, has been introduced with an aim to avoid local minima. This modification also helps in improving the convergence speed in some cases. Conditions for achieving global minimum for these kind of algorithms have been studied in detail. The performances of the proposed algorithms are compared with BP algorithm and extended Kalman filtering (EKF) on three bench-mark function approximation problems: XOR, 3-bit parity, and 8-3 encoder. The comparisons are made in terms of number of learning iterations and computational time required for convergence. It is found that the proposed algorithms (LF I and II) are much faster in convergence than other two algorithms to attain same accuracy. Finally, the comparison is made on a complex two-dimensional (2-D) Gabor function and effect of adaptive learning rate for faster convergence is verified. In a nutshell, the investigations made in this paper help us better understand the learning procedure of feedforward neural networks in terms of adaptive learning rate, convergence speed, and local minima.     

Machine fusion to enhance the topology preservation of vector quantization artificial neural networks

Artificial neural networks techniques have been successfully applied in vector quantization (VQ) encoding. The objective of VQ is to statistically preserve the topological relationships existing in a data set and to project the data to a lattice of lower dimensions, for visualization, compression, storage, or transmission purposes. However, one of the major drawbacks in the application of artificial neural networks is the difficulty to properly specify the structure of the lattice that best preserves the topology of the data. To overcome this problem, in this paper we introduce merging algorithms for machine-fusion, boosting-fusion-based and hybrid-fusion ensembles of SOM, NG and GSOM networks. In these ensembles not the output signals of the base learners are combined, but their architectures are properly merged. We empirically show the quality and robustness of the topological representation of our proposed algorithm using both synthetic and real benchmarks datasets.     

A novel image compression using grey models on a dynamic window

Based on the grey model (GM), a simple and fast methodology is developed for lossy image compression. First of all, the image is decomposed into some different-size image windows through the judgement of grey difference level; then the GM (1,1) of grey system theory is used as a fitter to model those window pixels. The proposed algorithms can be contrasted with the conventional compression techniques such as discrete cosine transform or vector quantization (VQ) algorithms in their dynamic modelling sequence and flexible block size. Especially, the compression and decompression process do not require an extra decoder and only utilize the modelling parameters to reconstruct the image by reversing the operation of GM (1,1). Experiments with some (512 x 512) images indicate that not only the average bit number per pixel and peak signal-to-noise ratio but also the coding time and decoding time of this lossy image compression algorithm based on GM (1,1) are better than those of block truncation coding with VQ.     

基于等级的无线传感网自适应分簇算法

为解决现有无线传感器网络（WSN）分簇算法难以同时兼顾其异构性和移动性,从而引发网络寿命较短、网络数据吞吐量较低等问题,提出了基于节点等级的自适应分簇算法。该算法按轮运行,每轮分为自适应分簇、簇建立、数据传输三个阶段。为解决节点移动性引发的簇首数目和成簇规模不合理的问题,在自适应分簇阶段,根据子区域内节点数目变化对相应子区域进行细化或就近合并,以确保每个子区域内节点数目在合理范围内。在簇建立阶段,选举簇内等级最高的节点为簇首,解决异构性引发的部分节点能耗过快、网络寿命缩短的问题;节点等级除考虑节点剩余能量外,还结合WSN实际应用,由节点剩余能量、能量消耗速率、到基站的距离、到簇内其他节点的距离综合决定。基于OMNeT++和Matlab的仿真实验结果表明,在节点移动速度为0~0.6 m/s的能量异构WSN环境下,较移动低功耗自适应集簇分层（LEACH-Mobile）算法和分布式能量有效分簇（DEEC）算法,运用所提算法分簇的WSN寿命延长了30.9%以上,网络数据吞吐量是其他两种算法分簇的网络的1.15倍以上。     

ANASA-a stochastic reinforcement algorithm for real-valued neuralcomputation

This paper introduces ANASA (adaptive neural algorithm of stochastic activation), a new, efficient, reinforcement learning algorithm for training neural units and networks with continuous output. The proposed method employs concepts, found in self-organizing neural networks theory and in reinforcement estimator learning algorithms, to extract and exploit information relative to previous input pattern presentations. In addition, it uses an adaptive learning rate function and a self-adjusting stochastic activation to accelerate the learning process. A form of optimal performance of the ANASA algorithm is proved (under a set of assumptions) via strong convergence theorems and concepts. Experimentally, the new algorithm yields results, which are superior compared to existing associative reinforcement learning methods in terms of accuracy and convergence rates. The rapid convergence rate of ANASA is demonstrated in a simple learning task, when it is used as a single neural unit, and in mathematical function modeling problems, when it is used to train various multilayered neural networks.     

一种基于改进CP网络与HMM相结合的混合音素识别方法

提出了一种基于改进对偶传播（ＣＰ）神经网络与隐驰尔可夫模型（ＨＭＭ）相结合的混合音素识别方法．这一方法的特点是用一个具有有指导学习矢量量化（ＬＶＱ）和动态节点分配等特性的改进的ＣＰ网络生成离散ＨＭＭ音素识别系统中的码书。因此,用这一方法构造的混合音素识别系统中的码书实际上是一个由有指导ＬＶＱ算法训练的具有很强分类能力的高性能分类器,这就意味着在用ＨＭＭ对语音信号进行建模之前,由码书产生的观测序列中     

Compression of 3-D Point Visual Data Using Vector Quantization and Rate-Distortion Optimization

In this paper, we propose adaptive and flexible quantization and compression algorithms for 3-D point data using vector quantization (VQ) and rate-distortion (R-D) optimization. The point data are composed of the position and the radius of sphere based on QSplat representation. The positions of child spheres are first transformed to the local coordinate system, which is determined by the parent-children relationship. The local coordinate transform makes the positions more compactly distributed in 3-D space, facilitating an effective application of VQ. We also develop a constrained encoding method for the radius data, which can provide a hole-free surface rendering at the decoder side. Furthermore, R-D optimized compression algorithm is proposed in order to allocate an optimal bitrate to each sphere. Experimental results show that the proposed algorithm can effectively compress the original 3-D point geometry at various bitrates.     

Adaptive recurrent neural network control using a structure adaptation algorithm

This paper proposes an adaptive recurrent neural network control (ARNNC) system with structure adaptation algorithm for the uncertain nonlinear systems. The developed ARNNC system is composed of a neural controller and a robust controller. The neural controller which uses a self-structuring recurrent neural network (SRNN) is the principal controller, and the robust controller is designed to achieve L ² tracking performance with desired attenuation level. The SRNN approximator is used to online estimate an ideal tracking controller with the online structuring and parameter learning algorithms. The structure learning possesses the ability of both adding and pruning hidden neurons, and the parameter learning adjusts the interconnection weights of neural network to achieve favorable approximation performance. And, by the L ² control design technique, the worst effect of approximation error on the tracking error can be attenuated to be less or equal to a specified level. Finally, the proposed ARNNC system with structure adaptation algorithm is applied to control two nonlinear dynamic systems. Simulation results prove that the proposed ARNNC system with structure adaptation algorithm can achieve favorable tracking performance even unknown the control system dynamics function.     

Efficient parallel processing of competitive learning algorithms

Vector quantization (VQ) is an attractive technique for lossy data compression, which has been a key technology for data storage and/or transfer. So far, various competitive learning (CL) algorithms have been proposed to design optimal codebooks presenting quantization with minimized errors. Although algorithmic improvements of these CL algorithms have achieved faster codebook design than conventional ones, limitations of speedup still exist when large data sets are processed on a single processor. Considering a variety of CL algorithms, parallel processing on flexible computing environment, like general-purpose parallel computers is in demand for a large-scale codebook design. This paper presents a formulation for efficiently parallelizing CL algorithms, suitable for distributed-memory parallel computers with a message-passing mechanism. Based on this formulation, we parallelize three CL algorithms: the Kohonen learning algorithm, the MMPDCL algorithm and the LOJ algorithm. Experimental results indicate a high scalability of the parallel algorithms on three different types of commercially available parallel computers: IBM SP2, NEC AzusA and PC cluster.     

An adaptive algorithm for resolving processor thrashing in load distribution

Processor thrashing in load distribution refers to the situation when a number of nodes try to negotiate with the same target node simultaneously. The performance of dynamic load-balancing algorithms can be degraded because processor thrashing can lead to receiver node overdrafting, thus causing congestion at a receiver node and reduction of workload distribution. In the paper we present an adaptive algorithm for resolving processor thrashing in load distribution. The algorithm is based on the integration of three components: (1) a batch task assignment policy, which allows a number of tasks to be transferred as a single batch from a sender to a receiver; (2) a negotiation protocol to obtain mutual agreement between a sender and a receiver on the batch size; and (3) an adaptive symmetrically-initiated location policy to select a potential transfer partner. Simulations reveal that our algorithm provides a significant performance improvement at high system loads because the algorithm can avoid processor thrashing so that CPU capacity is more fully utilized.     

Bi-modal derivative adaptive activation function sigmoidal feedforward artificial neural networks

In this work an adaptive mechanism for choosing the activation function is proposed and described. Four bi-modal derivative sigmoidal adaptive activation function is used as the activation function at the hidden layer of a single hidden layer sigmoidal feedforward artificial neural networks. These four bi-modal derivative activation functions are grouped as asymmetric and anti-symmetric activation functions (in groups of two each). For the purpose of comparison, the logistic function (an asymmetric function) and the function obtained by subtracting 0.5 from it (an anti-symmetric) function is also used as activation function for the hidden layer nodes’. The resilient backpropagation algorithm with improved weight-tracking (iRprop⁺) is used to adapt the parameter of the activation functions and also the weights and/or biases of the sigmoidal feedforward artificial neural networks. The learning tasks used to demonstrate the efficacy and efficiency of the proposed mechanism are 10 function approximation tasks and four real benchmark problems taken from the UCI machine learning repository. The obtained results demonstrate that both for asymmetric as well as anti-symmetric activation usage, the proposed/used adaptive activation functions are demonstratively as good as if not better than the sigmoidal function without any adaptive parameter when used as activation function of the hidden layer nodes.     

Large Scale Nonlinear Control System Fine-Tuning Through Learning

Despite the continuous advances in the fields of intelligent control and computing, the design and deployment of efficient large scale nonlinear control systems (LNCSs) requires a tedious fine-tuning of the LNCS parameters before and during the actual system operation. In the majority of LNCSs the fine-tuning process is performed by experienced personnel based on field observations via experimentation with different combinations of controller parameters, without the use of a systematic approach. The existing adaptive/neural/fuzzy control methodologies cannot be used towards the development of a systematic, automated fine-tuning procedure for general LNCS due to the strict assumptions they impose on the controlled system dynamics; on the other hand, adaptive optimization methodologies fail to guarantee an efficient and safe performance during the fine-tuning process, mainly due to the fact that these methodologies involve the use of random perturbations. In this paper, we introduce and analyze, both by means of mathematical arguments and simulation experiments, a new learning/adaptive algorithm that can provide with convergent, an efficient and safe fine-tuning of general LNCS. The proposed algorithm consists of a combination of two different algorithms proposed by Kosmatopoulos (2007 and 2008) and the incremental-extreme learning machine neural networks (I-ELM-NNs). Among the nice properties of the proposed algorithm is that it significantly outperforms the algorithms proposed by Kosmatopoulos as well as other existing adaptive optimization algorithms. Moreover, contrary to the algorithms proposed by Kosmatopoulos , the proposed algorithm can operate efficiently in the case where the exogenous system inputs (e.g., disturbances, commands, demand, etc.) are unbounded signals.