共查询到20条相似文献,搜索用时 0 毫秒
1.
One of the major drawbacks of the backpropagation algorithm is its slow rate of convergence. Researchers have tried several different approaches to speed up the convergence of backpropagation learning. In this paper, we present those rapid learning methods as three categories, and implement the representative methods of each category: (1) for the numerical method based approach, the Aitken's 2 process, (2) for the heuristics based approach, the dynamic adaptation of learning rate, and (3) for the learning strategy based approach, the selective presentation of learning samples. Based on these implementations, the performance is evaluated with experiments and the merits and demerits are briefly discussed.This work was supported in part by a grant from the Korea Science and Engineering Foundation (KOSEF) and the Center for Artificial Intelligence Research (CAIR), the Engineering Research Center (ERC) of Excellence Program. 相似文献
2.
3.
Dundar MM Fung G Krishnapuram B Rao RB 《IEEE transactions on bio-medical engineering》2008,55(3):1015-1021
Many computer-aided diagnosis (CAD) problems can be best modelled as a multiple-instance learning (MIL) problem with unbalanced data, i.e., the training data typically consists of a few positive bags, and a very large number of negative instances. Existing MIL algorithms are much too computationally expensive for these datasets. We describe CH, a framework for learning a Convex Hull representation of multiple instances that is significantly faster than existing MIL algorithms. Our CH framework applies to any standard hyperplane-based learning algorithm, and for some algorithms, is guaranteed to find the global optimal solution. Experimental studies on two different CAD applications further demonstrate that the proposed algorithm significantly improves diagnostic accuracy when compared to both MIL and traditional classifiers. Although not designed for standard MIL problems (which have both positive and negative bags and relatively balanced datasets), comparisons against other MIL methods on benchmark problems also indicate that the proposed method is competitive with the state-of-the-art. 相似文献
4.
Competitive learning algorithms for robust vector quantization 总被引:1,自引:0,他引:1
The efficient representation and encoding of signals with limited resources, e.g., finite storage capacity and restricted transmission bandwidth, is a fundamental problem in technical as well as biological information processing systems. Typically, under realistic circumstances, the encoding and communication of messages has to deal with different sources of noise and disturbances. We propose a unifying approach to data compression by robust vector quantization, which explicitly deals with channel noise, bandwidth limitations, and random elimination of prototypes. The resulting algorithm is able to limit the detrimental effect of noise in a very general communication scenario. In addition, the presented model allows us to derive a novel competitive neural networks algorithm, which covers topology preserving feature maps, the so-called neural-gas algorithm, and the maximum entropy soft-max rule as special cases. Furthermore, continuation methods based on these noise models improve the codebook design by reducing the sensitivity to local minima. We show an exemplary application of the novel robust vector quantization algorithm to image compression for a teleconferencing system 相似文献
5.
It is wellknown that gradient search fails in adaptive IIR filters, since their mean-square error surfaces may be multi-modal. In the letter a new approach based on learning algorithms is shown to be capable of performing global optimisation. The new algorithms are suitable for both adaptive FIR and IIR filters.<> 相似文献
6.
Potamianos G. Goutsias J. 《IEEE transactions on information theory / Professional Technical Group on Information Theory》1997,43(6):1948-1965
We present an analysis of previously proposed Monte Carlo algorithms for estimating the partition function of a Gibbs random field. We show that this problem reduces to estimating one or more expectations of suitable functionals of the Gibbs states with respect to properly chosen Gibbs distributions. As expected, the resulting estimators are consistent. Certain generalizations are also provided. We study computational complexity with respect to grid size and show that Monte Carlo partition function estimation algorithms can be classified into two categories: E-type algorithms that are of exponential complexity and P-type algorithms that are of polynomial complexity, Turing reducible to the problem of sampling from the Gibbs distribution. E-type algorithms require estimating a single expectation, whereas, P-type algorithms require estimating a number of expectations with respect to Gibbs distributions which are chosen to be sufficiently “close” to each other. In the latter case, the required number of expectations is of polynomial order with respect to grid size. We compare computational complexity by using both theoretical results and simulation experiments. We determine the most efficient E-type and P-type algorithms and conclude that P-type algorithms are more appropriate for partition function estimation. We finally suggest a practical and efficient P-type algorithm for this task 相似文献
7.
Guang Deng 《Signal Processing, IEEE Transactions on》2004,52(8):2286-2297
In this paper, we consider a signal/parameter estimation problem that is based on a linear model structure and a given setting of statistical models with unknown hyperparameters. We consider several combinations of Gaussian and Laplacian models. We develop iterative algorithms based on two typical machine learning methods - the evidence-based method and the integration-based method - to deal with the hyperparameters. We have applied the proposed algorithms to adaptive prediction and wavelet denoising. In linear prediction, we show that the proposed algorithms are efficient tools for tackling a difficult problem of adapting simultaneously the order and the coefficients of the predictor. In wavelet denoising, we show that by using the proposed algorithms, the noisy wavelet coefficients are subject to shrinkage and thresholding. 相似文献
8.
9.
On the generalization ability of on-line learning algorithms 总被引:3,自引:0,他引:3
Cesa-Bianchi N. Conconi A. Gentile C. 《IEEE transactions on information theory / Professional Technical Group on Information Theory》2004,50(9):2050-2057
In this paper, it is shown how to extract a hypothesis with small risk from the ensemble of hypotheses generated by an arbitrary on-line learning algorithm run on an independent and identically distributed (i.i.d.) sample of data. Using a simple large deviation argument, we prove tight data-dependent bounds for the risk of this hypothesis in terms of an easily computable statistic M/sub n/ associated with the on-line performance of the ensemble. Via sharp pointwise bounds on M/sub n/, we then obtain risk tail bounds for kernel perceptron algorithms in terms of the spectrum of the empirical kernel matrix. These bounds reveal that the linear hypotheses found via our approach achieve optimal tradeoffs between hinge loss and margin size over the class of all linear functions, an issue that was left open by previous results. A distinctive feature of our approach is that the key tools for our analysis come from the model of prediction of individual sequences; i.e., a model making no probabilistic assumptions on the source generating the data. In fact, these tools turn out to be so powerful that we only need very elementary statistical facts to obtain our final risk bounds. 相似文献
10.
宁一璇 《智能计算机与应用》2021,11(5):6-12
随着机器学习算法在人工智能各领域的广泛应用,人们开始关注机器学习算法的质量分析.由于机器学习算法中缺少测试语言,对学习算法进行质量分析是很困难的,基于此,本文提出了基于不变量的机器学习算法分析方法,对5种机器学习算法进行不同参数下的不变量生成,得到不变量集合.通过动态筛选机制和函数调用图,对不变量集合进行筛选和提取,得... 相似文献
11.
肖行 《智能计算机与应用》2021,11(3):215-216,封3
深度学习技术的运用正日趋广泛,深度学习自身的高效性和智能性受到研究者的青睐.通过对深度学习影像分类的剖析,进一步探究深度学习在影像识别方向的应用,介绍了主要用于影像分类识别的基于深度学习的医疗影像检测算法,可作为开展深度学习技术运用于医学影像检测研究工作的有益参考. 相似文献
12.
A constructive learning algorithm is used to generate networks that learn to approximate the functional of the magnetotelluric inverse problem. Based on synthetic data, several experiments are performed in order to generate and test the neural networks. Rather than producing, at the present time, a practical algorithm using this approach, the object of the paper is to explore the possibilities offered by the new tools. The generated networks can be used as an internal module in a more general inversion program, or their predicted models can be used by themselves or simply as inputs to an optimization program 相似文献
13.
Shah S. Sastry P.S. 《IEEE transactions on systems, man and cybernetics. Part C, Applications and reviews》1999,29(4):494-505
We present methods for learning and pruning oblique decision trees. We propose a new function for evaluating different split rules at each node while growing the decision tree. Unlike the other evaluation functions currently used in the literature (which are all based on some notion of purity of a node), this new evaluation function is based on the concept of degree of linear separability. We adopt a correlation based optimization technique called the Alopex algorithm (K.P. Unnikrishnaan and K.P. Venugopal, 1994) for finding the split rule that optimizes our evaluation function at each node. The algorithm we present is applicable only for 2-class problems. Through empirical studies, we demonstrate that our algorithm learns good compact decision trees. We suggest a representation scheme for oblique decision trees that makes explicit the fact that an oblique decision tree represents each class as a union of convex sets bounded by hyperplanes in the feature space. Using this representation, we present a new pruning technique. Unlike other pruning techniques, which generally replace heuristically selected subtrees of the original tree by leaves, our method can radically restructure the decision tree. Through empirical investigation, we demonstrate the effectiveness of our method 相似文献
14.
以三氯蔗糖、赤藓糖醇、木糖醇3种人工甜味剂为研究对象,采用太赫兹时域光谱技术,结合多种机器学习和优化算法对甜味剂与面粉混合物的光谱数据进行系统的分类识别和定量回归研究。结果表明,麻雀搜索算法-支持向量机模型/支持向量回归模型(SSA-SVM/SVR)对混合物的定性及定量分析结果均达到最优,分类预测的准确率达到95.56%,定量回归预测的最佳回归系数R 2 ![]()
![]()
为0.999 8,实现了3种甜味剂和面粉混合物的高精确度分类和定量分析,为人工甜味剂的快速检测提供了一种有效可靠的新思路。 相似文献
15.
图像是信息的重要承载形式。雾霾的出现降低了图像采集设备采集到的图像质量,容易出现色彩暗淡、对比度和饱和度降低、细节信息丢失等问题,直接影响了有用信息的表达和利用。目前对图像去雾的研究多采用深度学习的方法,卷积神经网络代替了人工特征提取方式,取得了优于传统算法的去雾效果,但普遍存在着对真实世界雾霾图像和清晰图像对的依赖。无监督学习的方法带来了新的解决思路。从监督学习和无监督学习的角度对有代表性的深度学习图像去雾算法进行分类,归纳了常用的数据集、评价指标,概括分析了有影响力的去雾模型的核心思想,总结了各算法的优缺点和适用场景。针对目前工作存在的不足,探索了下一步研究的方向。 相似文献
16.
17.
提出了一种新的基于图像块距离的邻域选择方法,并将其应用于流形学习中,得到一类新的高光谱图像非线性降维算法。该类算法利用高光谱图像物理特性,结合图像的光谱信息和空间信息,在最大限度减小图像信息冗余的基础之上,很好地保持了原始数据集的特性。与其它高光谱图像的降维算法相比,改进的流形学习算法不仅考虑到高光谱图像本身的空间关系,而且利用图像块距离更好地保持了数据点之间的局部特性,从而有效地去除原始数据集光谱维和空间维的冗余信息。实际高光谱数据的实验结果表明,所提出的算法在应用于高光谱图像分类时,与其它方法相比具有更高的分类精度。 相似文献
18.
Neural network constructive algorithms: Trading generalization for learning efficiency? 总被引:5,自引:0,他引:5
F. J. Śmieja 《Circuits, Systems, and Signal Processing》1993,12(2):331-374
There are currently several types of constructive, (or growth), algorithms available for training a feed-forward neural network. This paper describes and explains the main ones, using a fundamental approach to the multi-layer perceptron problem-solving mechanisms. The claimed convergence properties of the algorithms are verified using just two mapping theorems, which consequently enables all the algorithms to be unified under a basic mechanism. The algorithms are compared and contrasted and the deficiencies of some highlighted. The fundamental reasons for the actual success of these algorithms are extracted, and used to suggest where they might most fruitfully be applied. A suspicion that they are not a panacea for all current neural network difficulties, and that one must somewhere along the line pay for the learning efficiency they promise, is developed into an argument that their generalization abilities will lie on average below that of back-propagation.Funded by the German Ministry of Research and Technology, grant number 01 IN 111 A/4.German National Research Centre for Computer Science (GMD), Schloß Birlinghoven, 5205 St. Augustin 1, Germany. 相似文献
19.
Stochastic gradient algorithms for design of minimum error-rate linear dispersion codes in MIMO wireless systems 总被引:1,自引:0,他引:1
Xiaodong Wang Krishnamurthy V. Jibing Wang 《Signal Processing, IEEE Transactions on》2006,54(4):1242-1255
Linear dispersion (LD) codes are a good candidate for high-data-rate multiple-input multiple-ouput (MIMO) signaling. Traditionally LD codes were designed by maximizing the average mutual information, which cannot guarantee good error performance. This paper presents a new design scheme for LD codes that directly minimizes the block error rate (BLER) in MIMO channels with arbitrary fading statistics and various detection algorithms. For MIMO systems employing LD codes, the error rate does not admit an explicit form. Therefore, we cannot use deterministic optimization methods to design the minimum-error-rate LD codes. In this paper, we propose a simulation-based optimization methodology for the design of LD codes through stochastic approximation and simulation-based gradient estimation. The gradient estimation is done using the score function method originally developed in the discrete-event-system community. The proposed method can be applied to design the minimum-error-rate LD codes for a variety of detector structures including the maximum-likelihood (ML) detector and several suboptimal detectors. It can also design optimal codes under arbitrary fading channel statistics; in particular, it can take into account the knowledge of spatial fading correlation at the transmitter and receiver ends. Simulation results show that codes generated by the proposed new design paradigm generally outperform the codes designed based on algebraic number theory. 相似文献
20.
This paper evaluates a segmentation technique for magnetic resonance (MR) images of the brain based on fuzzy algorithms for learning vector quantization (FALVQ). These algorithms perform vector quantization by updating all prototypes of a competitive network through an unsupervised learning process. Segmentation of MR images is formulated as an unsupervised vector quantization process, where the local values of different relaxation parameters form the feature vectors which are represented by a relatively small set of prototypes. The experiments evaluate a variety of FALVQ algorithms in terms of their ability to identify different tissues and discriminate between normal tissues and abnormalities. 相似文献