基于权值拟熵的神经网络剪枝算法研究

“剪枝算法”是一种通过简化神经网络结构来避免网络过拟合的有效方法之一。将权值拟熵作为惩罚项加入目标函数中,使多层前向神经网络在学习过程中自动约束权值分布,并以权值敏感度作为简化标准,避免了单纯依赖权值大小剪枝的随机性。由于在剪枝过程中只剪去数值小并且敏感度低的连接权,所以网络简化后不需要重新训练,算法效率明显提高。仿真结果证明上述方法算法简单易行,并且对前向神经网络的泛化能力有较好的改善作用。     

Machine Learning-Based Pruning Technique for Low Power Approximate Computing

Approximate Computing is a low power achieving technique that offers an additional degree of freedom to design digital circuits. Pruning is one of the types of approximate circuit design technique which removes logic gates or wires in the circuit to reduce power consumption with minimal insertion of error. In this work, a novel machine learning (ML) -based pruning technique is introduced to design digital circuits. The machine-learning algorithm of the random forest decision tree is used to prune nodes selectively based on their input pattern. In addition, an error compensation value is added to the original output to reduce an error rate. Experimental results proved the efficiency of the proposed technique in terms of area, power and error rate. Compared to conventional pruning, proposed ML pruning achieves 32% and 26% of the area and delay reductions in 8*8 multiplier implementation. Low power image processing algorithms are essential in various applications like image compression and enhancement algorithms. For real-time evaluation, proposed ML optimized pruning is applied in discrete cosine transform (DCT). It is a basic element of image and video processing applications. Experimental results on benchmark images show that proposed pruning achieves a very good peak signal-to-noise ratio (PSNR) value with a considerable amount of energy savings compared to other methods.     

An Optimization Methodology for Neural Network Weights and Architectures

This paper introduces a methodology for neural network global optimization. The aim is the simultaneous optimization of multilayer perceptron (MLP) network weights and architectures, in order to generate topologies with few connections and high classification performance for any data sets. The approach combines the advantages of simulated annealing, tabu search and the backpropagation training algorithm in order to generate an automatic process for producing networks with high classification performance and low complexity. Experimental results obtained with four classification problems and one prediction problem has shown to be better than those obtained by the most commonly used optimization techniques     

A New Poisson Noise Filter Based on Weights Optimization

We propose a new image denoising algorithm when the data is contaminated by a Poisson noise. As in the Non-Local Means filter, the proposed algorithm is based on a weighted linear combination of the observed image. But in contrast to the latter where the weights are defined by a Gaussian kernel, we propose to choose them in an optimal way. First some “oracle” weights are defined by minimizing a very tight upper bound of the Mean Square Error. For a practical application the weights are estimated from the observed image. We prove that the proposed filter converges at the usual optimal rate to the true image. Simulation results are presented to compare the performance of the presented filter with conventional filtering methods.     

基于双角度并行剪枝的VGG 16优化方法

近年来,对卷积神经网络的轻量化工作更多的是根据滤波器的范数值来进行裁剪,范数值越小,裁剪之后对网络的影响就越小.这种思路充分利用了滤波器的数值特性,但也忽略了滤波器的结构特性.基于上述观点,文中尝试将凝聚层次聚类算法AHCF(Agglomerative Hierarchical Clustering Method fo...     

A Neuro-Genetic Algorithm for Heteroskedastic Time-Series Processes Empirical Tests on Global Asset Returns

 The paper proposes a new neuro-genetic hybrid algorithm (NGHA) for coping with ill-conditioned time-series processes. Extensive testing and comparisons to various heteroskedastic models indicate that the neuro-genetic algorithm may be a useful device for modelling complicated time series. NGHA is used to model a factor price series corresponding to the European factor of a representative set of global asset returns. NGHA provides a platform for adapting evolutionary computation to the search for suitable networks for observed time series.     

基于光顺优化的NURBS曲线权因子估计方法

基于弹性均匀细梁的应力能和扰动我顺优化,提出了ＮＵＲＢＳ曲线权因子的一种估计方法,将应力能和扰动能离散化,得到一个泛函,对此泛函极小化而得权因子,同时,文中给出的极小泛函统一了目前文献中出现的一类不同光顺标准。为了减少导数的计算量,文中采用了ＮＵＲＢＳ曲线的矩阵表示,最后,通过数值算例比较了各种光顺标准的优劣。     

一种具有初始化功能的自适应惯性权重粒子群算法

提出了一种改进的具有初始化功能的自适应惯性权重粒子群优化(PSO)算法.该算法首先引入自适应惯性权重策略均衡全局和局部搜索能力,并针对运行过程中出现停滞现象的粒子群,围绕其加权重心位置重新初始化,引导粒子突破了局部极值的限制,提高了算法的收敛速度.最后,将此算法、PSO算法及惯性权重线性递减的PSO(LDW-PSO)算法进行了比较.实验结果表明,该算法不仅有效地增强了粒子突破局部极值的能力,而且算法的收敛速度和稳定性也有了一定的提高.     

属性赋权的K-Modes算法优化

传统K-Modes算法的一个主要问题是属性选择问题。K-Modes算法在聚类过程中对每一个属性都同等看待,而在实际应用中,很多数据集仅有几个重要属性对聚类起作用。为了考虑不同属性对聚类的不同影响,将K-Modes聚类算法与属性权重的最优化结合起来,提出一种属性自动赋权的FW-K-Modes算法。该算法不仅可以提高传统K-Modes聚类算法的聚类精度,还能分析各维属性对聚类的贡献程度,实现关键属性的选择。对多个UCI数据集进行了实验,验证了该算法的优良特性。     

CDC与REP结合的决策树剪枝优化算法

为改善剪枝算法单一的事前剪枝或事后剪枝导致分类响应时间长、准确度低的问题,在REP事后剪枝的基础上,提出一种CDC与REP结合的决策树剪枝优化算法。使用CDC算法在生成决策树的同时,利用左右子树节点差异比来排除部分非叶子节点,决策树生成后再通过REP算法对决策树进一步剪枝。实验结果表明,该算法可避免庞大决策树的生成过程过于细化导致过于拟合的现象,与其他算法相比,能减少分裂时间,提高决策树分裂的正确率。     

基于剪枝技术的自适应PPM预测模型

建立有效的用户浏览预测模型,对用户的浏览行为进行准确的预测,是Web预取的关键。标准PPM预测模型由于存在存储复杂度高、执行效率低等缺点,影响了其推广和应用。文章基于剪枝技术,依据Zipf法则及Web对象访问特征对标准PPM预测模型进行预先剪枝和后剪枝,构造出一种自适应PPM预测模型。实验表明,该模型不仅能动态预测用户的Web浏览特征,而且在预测准确率和存储复杂度方面都有一定程度的提高。     

融合Sin混沌和分段权值的阿基米德优化算法

针对阿基米德优化算法（Archimedes optimization algorithm,AOA）存在全局搜索能力弱、收敛精度低,易陷入局部最优等问题,提出融合Sin混沌和分段权值的阿基米德优化算法（SAOA）。采用无限折叠迭代的Sin混沌反向学习策略初始化种群,提高初始阶段解的质量,为全局搜索多样性奠定基础;引入算数交叉算子,将当前个体向与全局最优个体进行交叉,引导种群向最优解区域寻优,提高全局搜索能力;引入分段权值策略,平衡算法的全局勘探与局部开发能力,降低算法陷入局部最优的概率;通过对8个测试函数和部分CEC2014函数进行仿真实验及Wilcoxon秩和检验来评估改进算法的寻优性能,实验结果表明改进算法在搜索精度、收敛速度和稳定性等方面均有较大提升。另外,引入优化机械设计案例进行测试分析,进一步验证SAOA在工程优化问题上的可行性和适用性。     

基于多级神经网络的盾构法隧道施工参数优化

在盾构法隧道施工中,合理地设置施工参数、确保地面沉降控制在一定的范围之内是实际施工中最为关注的问题.本文将盾构推进的过程划分成七个阶段,每个阶段用一个神经网络进行模拟,在此基础上构造成多级神经网络,拟合盾构法隧道施工中施工参数与地面沉降之间关系的数学模型. 最后根据保证质量、兼顾效益和效率的目标,采用遗传算法进行施工参数的优化匹配,并提出了相应的控制方案.该方法已在多项工程项目中得到使用. 结果表明,此方法对工程实际施工有很好的指导作用 .     

A Pruning Algorithm for Stable Voronoi Skeletons

We present a skeleton computation algorithm for binary image shape which is stable and efficient. The algorithm follows these steps: first the shape boundary curves are subsampled, then the Voronoi Skeleton is computed from the resulting reduced boundary set of points, and finally, a?novel two stage pruning procedure is applied to obtain a?simplified skeleton. The first stage removes skeleton edges non fully included in the shape. The second stage applies an enhanced variation of the Discrete Curve Evolution (DCE) for Voronoi skeletons. We obtain improved skeleton stability, complexity reduction and noise robustness. Pruning computing time efficiency is improved thanks to some properties of Voronoi skeletons. Entire skeleton edges can be removed or retained on the basis of conditions tested on the edge endpoints. Pattern recognition experiments and skeleton stability experiments of the algorithm outperform previous approaches in the literature.     

基于自适应权重的立体匹配优化算法

为解决现有立体匹配算法对低纹理以及视差不连续区域匹配效果较差的问题,提出一种改进的立体匹配优化算法。在传统自适应权重算法匹配代价的基础上,融合高斯差分图像差分信息,即左右图像高斯差分图的差分,重新定义其初始匹配代价,增加算法在视差不连续区域的鲁棒性,并加入边缘约束和视差边缘约束迭代聚类以及基于高斯差分图的自适应窗口算法,保证改进算法在低纹理区域的匹配性能,消除坏点与视差空洞。将该算法与传统自适应权重匹配算法分别在Middlebury数据集上进行匹配实验,结果表明,该算法平均性能提升了15.05%,明显优于传统自适应权重匹配算法。     

A Combinatorial Optimization Technique for the Sequential Decomposition of Erosions and Dilations

This paper presents a general algorithm for the automatic proof that an erosion (respectively, dilation) has a sequential decomposition or not. If the decomposition exists, an optimum decomposition is presented. The algorithm is based on a branch and bound search, with pruning strategies and bounds based on algebraic and geometrical properties deduced formally. This technique generalizes classical results as Zhuang and Haralick, Xu, and Park and Chin, with equivalent or improved performance. Finally, theoretical analysis of the proposed algorithm and experimental results are presented.     

基于思维进化算法的神经网络权值与结构优化

人工神经网络应用的关键在于权值和结构的优化。思维进化计算(MEC)是模拟人类思维进化过程的一种新的进化算法,具有极强的全局寻优能力,在数值优化和非数值优化方面均显示出明显的优越性。本文在思维进化计算框架的基础上,提出了一种用于人工神经网络权值与结构优化的思维进化算法,设计了有效的结构优化‘趋同’与‘异化’算
子;在局部范围内寻求局部最优解,然后使用异化算子跳出局部范围的约束,在整个解空间寻求全局最优解。仿真结果说明了方法的正确性与有效性。     

A New Multisection Technique in Interval Methods for Global Optimization

A new multisection technique in interval methods for global optimization is investigated, and numerical tests demonstrate that the efficiency of the underlying global optimization method can be improved substantially. The heuristic rule is based on experiences that suggest the subdivision of the current subinterval into a larger number of pieces only if it is located in the neighbourhood of a minimizer point. An estimator of the proximity of a subinterval to the region of attraction to a minimizer point is utilized. According to the numerical study made, the new multisection strategies seem to be indispensable, and can improve both the computational and the memory complexity substantially. Received May 31, 1999; revised January 20, 2000     

一种高质量的领域无关前向规划剪枝策略

前向启发式搜索和放宽规划方法被很多领域无关的规划器所采用,被认为是一种有效的规划范型.FF规划器利用放宽规划图计算状态的启发式估值,并提取有利动作集合进行前向搜索的剪枝.但过大的有利动作集合造成了过多的消耗.文中提出了一种新的高质量的领域无关剪枝策略.该策略根据放宽规划图的动作层和命题层之间的关系,提取出所谓的直接效用动作集合,此集合之外的其它动作都被剪枝.直接效用动作集合比FF的有利动作集合更加精简,更具启发性,能指导前向搜索集中在那些离目标更近的状态.根据直接效用动作作者开发了一种新的lookahead搜索邻居,并应用在改进后的增强型爬山搜索算法中,使得前向搜索具备良好的前瞻性.当增强型爬山法失败时,采取一种从局部极小值重启完备搜索的策略以保持系统完备性.通过对国际规划大赛基准问题的测试表明,基于该剪枝策略及前向搜索算法实现的前向规划系统有效地缩小了搜索空间,搜索的节点数目比FF的有利动作策略明显要少,搜索效率有显著的提升.     

A Growing and Pruning Method for Radial Basis Function Networks

A recently published generalized growing and pruning (GGAP) training algorithm for radial basis function (RBF) neural networks is studied and modified. GGAP is a resource-allocating network (RAN) algorithm, which means that a created network unit that consistently makes little contribution to the network's performance can be removed during the training. GGAP states a formula for computing the significance of the network units, which requires a d-fold numerical integration for arbitrary probability density function $p({bf x})$ of the input data ${bf x} ({bf x} in {{bf R}}^{d})$ . In this work, the GGAP formula is approximated using a Gaussian mixture model (GMM) for $p({bf x})$ and an analytical solution of the approximated unit significance is derived. This makes it possible to employ the modified GGAP for input data having complex and high-dimensional $p({bf x})$, which was not possible in the original GGAP. The results of an extensive experimental study show that the modified algorithm outperforms the original GGAP achieving both a lower prediction error and reduced complexity of the trained network.