新的软间隔AdaBoost弱分类器权重调整算法

为避免硬间隔算法过分强调较难分类样本而导致泛化性能下降的问题,提出一种新的基于软间隔的AdaBoost-QP算法。在样本硬间隔中加入松弛项,得到软间隔的概念,以优化样本间隔分布、调整弱分类器的权重。实验结果表明,该算法能降低泛化误差,提高 AdaBoost算法的泛化性能。     

Orthonormal Vector Sets Regularization with PDE's and Applications

We are interested in regularizing fields of orthonormal vector sets, using constraint-preserving anisotropic diffusion PDE's. Each point of such a field is defined by multiple orthogonal and unitary vectors and can indeed represent a lot of interesting orientation features such as direction vectors or orthogonal matrices (among other examples). We first develop a general variational framework that solves this regularization problem, thanks to a constrained minimization of -functionals. This leads to a set of coupled vector-valued PDE's preserving the orthonormal constraints. Then, we focus on particular applications of this general framework, including the restoration of noisy direction fields, noisy chromaticity color images, estimated camera motions and DT-MRI (Diffusion Tensor MRI) datasets.     

最优间隔分布脊回归

脊回归(ridge regression, RR)是经典的机器学习算法之一,广泛应用于人脸识别、基因工程等诸多领域.其具有优化目标凸、存在闭合解、可解释性强以及易于核化等优点,但是脊回归的优化目标并没有考虑样本之间的结构关系.监督流形正则化学习是最具代表性的、最成功的脊回归正则化方法之一,其通过最小化每类类内方差来考虑样本之间的类内结构关系,可是单纯地只考虑类内结构仍然不够全面.以一种全新的视角重新审视最近提出的“最优间隔分布学习”原理,发现了最优间隔分布的目标可以同时优化类内间隔方差和类间间隔方差,从而同时优化了局部的类内结构和全局的类间结构.基于此提出了一种充分考虑数据结构化特征的脊回归算法——最优间隔分布脊回归(optimal margin distribution machine ridge regression, ODMRR)算法,该算法具有RR以及MRRR(manifold regularization ridge regression)的各种优势.最后通过实验验证了该方法具有优越的性能.     

A polynomial algorithm for recognizing bounded cutwidth in hypergraphs

The Min Cut Linear Arrangement (Min Cut) or Backboard Permutation (BP) problem, where it is desired to minimize backplane area or cutwidth in hypergraphs, has a long history of interest. To determine, for given graphG and integerk, whetherG has cutwidth at mostk is known to beNP-complete even for planar graphs with maximum vertex degree 3. (As graphs are a special case of hypergraphs, it is alsoNP-complete for hypergraphs.) Recently, Cahoon and Sahni described O(n) and O(n ³) algorithms for determining if a hypergraph had cutwidth 1 and 2, respectively. However, for any fixedk>2, it remained open whether determining if an arbitrary hypergraph has cutwidth at mostk was in the classP. We show a positive answer; specifically, we describe an O(n ^m ) algorithm, withm=k ²+3k+3, which determines if a hypergraph withn vertices has cutwidthk.The work of Z. Miller was begun during the fall semester 1985 spent as a Visiting Professor in the Computer Science Program, University of Texas at Dallas. Support was also provided for this author by ONR Grant No. N00014-85-K-0621.     

Weakly Constrained Minimization: Application to the Estimation of Images and Signals Involving Constant Regions

We focus on the question of how the shape of a cost-function determines the features manifested by its local (and hence global) minimizers. Our goal is to check the possibility that the local minimizers of an unconstrained cost-function satisfy different subsets of affine constraints dependent on the data, hence the word weak. A typical example is the estimation of images and signals which are constant on some regions. We provide general conditions on cost-functions which ensure that their minimizers can satisfy weak constraints when noisy data range over an open subset. These cost-functions are non-smooth at all points satisfying the weak constraints. In contrast, the local minimizers of smooth cost-functions can almost never satisfy weak constraints. These results, obtained in a general setting, are applied to analyze the minimizers of cost-functions, composed of a data-fidelity term and a regularization term. We thus consider the effect produced by non-smooth regularization, in comparison with smooth regularization. In particular, these results explain the stair-casing effect, well known in total-variation methods. Theoretical results are illustrated using analytical examples and numerical experiments.     

Nonlinear multigrid methods for total variation image denoising

The classical image denoising technique introduced by Rudin, Osher, and Fatemi [17] a decade ago, leads to solve a constrained minimization problem for the total variation (TV) of the image. The formal first variation of the minimization problem is a nonlinear and highly anisotropic boundary value problem. In this paper, a computational PDE method based on a nonlinear multigrid scheme for restoring noisy images is suggested. Here, we examine different discretizations for the Euler–Lagrange equation as well as different smoothers within the multigrid scheme. Then we describe the iterative total variation regularization scheme, which starts with an isotropic (smooth) problem and leads to smooth edges in the image. Within the iteration the problem becomes more and more anisotropic and converges to an image with sharp edges. Finally, we present some experimental results for synthetic and real images.     

Incremental margin algorithm for large margin classifiers

In this contribution, we introduce a new on-line approximate maximal margin learning algorithm based on an extension of the perceptron algorithm. This extension, which we call fixed margin perceptron (FMP), finds the solution of a linearly separable learning problem given a fixed margin. It is shown that this algorithm converges in updates, where γ_f<γ^* is the fixed margin, γ^* is the optimum margin and R is the radius of the ball that circumscribes the data. The incremental margin algorithm (IMA) approximates the large margin solution by successively using FMP with increasing margin values. This incremental approach always guarantees a good solution at hands. Also, it is easy to implement and avoids quadratic programming methods. IMA was tested using several different data sets and it yields results similar to those found by an SVM.     

Regularized margin-based conditional log-likelihood loss for prototype learning

The classification performance of nearest prototype classifiers largely relies on the prototype learning algorithm. The minimum classification error (MCE) method and the soft nearest prototype classifier (SNPC) method are two important algorithms using misclassification loss. This paper proposes a new prototype learning algorithm based on the conditional log-likelihood loss (CLL), which is based on the discriminative model called log-likelihood of margin (LOGM). A regularization term is added to avoid over-fitting in training as well as to maximize the hypothesis margin. The CLL in the LOGM algorithm is a convex function of margin, and so, shows better convergence than the MCE. In addition, we show the effects of distance metric learning with both prototype-dependent weighting and prototype-independent weighting. Our empirical study on the benchmark datasets demonstrates that the LOGM algorithm yields higher classification accuracies than the MCE, generalized learning vector quantization (GLVQ), soft nearest prototype classifier (SNPC) and the robust soft learning vector quantization (RSLVQ), and moreover, the LOGM with prototype-dependent weighting achieves comparable accuracies to the support vector machine (SVM) classifier.     

On the VC Dimension of Bounded Margin Classifiers

In this paper we prove a result that is fundamental to the generalization properties of Vapnik's support vector machines and other large margin classifiers. In particular, we prove that the minimum margin over all dichotomies of k n + 1 points inside a unit ball in R ⁿ is maximized when the points form a regular simplex on the unit sphere. We also provide an alternative proof directly in the framework of level fat shattering.     

Wer gewinnt durch Offshoring?

Die Gartner Group geht davon aus, dass 2003 etwa die Hälfte der ausgelagerten Projekte von den Auftraggebenden als nicht erfolgreich bewertet werden. Esther Ruiz Ben, Regina Claus S. 34

---

Although the country did outsource more IT customer support positions than any other job type, German business displayed a higher need to export software development and testing positions than it did for tech support. Jarad Carleton, Frost & Sullivan

     

Unconstrained discrete infmax problem solving applied to yield design

One of the methods of solving unconstrained discrete infmax or minimax problems consists in regularizing the functionF(x)=max _i f _i (x),i=1,...,m, mR ⁿ using various techniques. A new method of solving these problems, which is similar in nature to the regularization method, is presented. It is, however, differentiated from the latter by the fact that regularization is not applied toF(x) but to a function parametered byp (p1), the expression of which does not contain the max operator. Depending on the value ofp, regularization is either local (p=1) or total (p>1).The practical advantage of the proposed method is highlighted in solving large scale problems arising from the static yield design method.     

PAC Analogues of Perceptron and Winnow Via Boosting the Margin

We describe a novel family of PAC model algorithms for learning linear threshold functions. The new algorithms work by boosting a simple weak learner and exhibit sample complexity bounds remarkably similar to those of known online algorithms such as Perceptron and Winnow, thus suggesting that these well-studied online algorithms in some sense correspond to instances of boosting. We show that the new algorithms can be viewed as natural PAC analogues of the online p-norm algorithms which have recently been studied by Grove, Littlestone, and Schuurmans (1997, Proceedings of the Tenth Annual Conference on Computational Learning Theory (pp. 171–183) and Gentile and Littlestone (1999, Proceedings of the Twelfth Annual Conference on Computational Learning Theory (pp. 1–11). As special cases of the algorithm, by taking p = 2 and p = we obtain natural boosting-based PAC analogues of Perceptron and Winnow respectively. The p = case of our algorithm can also be viewed as a generalization (with an improved sample complexity bound) of Jackson and Craven's PAC-model boosting-based algorithm for learning sparse perceptrons (Jackson & Craven, 1996, Advances in neural information processing systems 8, MIT Press). The analysis of the generalization error of the new algorithms relies on techniques from the theory of large margin classification.     

On-line Scheduling for Jobs with Arbitrary Release Times

This paper considers the problem of on-line scheduling a list of independent jobs in which each job has an arbitrary release time on m parallel identical machines. A tight bound is given for List Scheduling(LS) algorithm and a better algorithm is given for m2.AMS Subject Classifications: 90B35 (90C27).This research is supported by Singapore-MIT Alliance.     

稀疏性正则化的图像Laplace去噪及PR算子分裂算法*

在Bayesian-MAP框架下,建立了针对Laplace噪声的稀疏性正则化图像去噪凸变分模型,模型采用L1范数作为数据保真项,非光滑的正则项约束图像在过完备字典下表示系数的稀疏性。进一步基于Peaceman-Rachford算子分裂算法,提出了数值求解该非光滑模型的多步迭代快速算法,通过引入保真项与稀疏性正则项的邻近算子,可将原问题转换为两个简单子问题的迭代求解,降低了计算复杂性。实验结果验证了模型与数值算法的有效性,本算法在摄像自动报靶系统中得到了应用。     

A Geometric Approach for Regularization of the Data Term in Stereo-Vision

Every stereovision application must cope with the correspondence problem. The space of the matching variables, often consisting of spatial coordinates, intensity and disparity, is commonly referred as the data term (space). Since the data is often noisy a-priori, preference is required to result a smooth disparity (or piecewise smooth). To this end, each local method (e.g. window correlation techniques) performs a regularization of the data space. In this paper we propose a geometric framework for anisotropic regularization of the data space seeking to preserve the discontinuities in this space when filtering out the noise. On the other hand, the global methods consider a non-regularized data term with a smoothing constraint imposed directly on the disparity. This paper also proposes a new idea where the data space is regularized in a global method prior to the disparity evaluation. The idea is implemented on the state of the art variational method. Experimental results on the Middlebury real images demonstrate the advantages of the proposed approach.

Problems of complex system modeling and applied mathematical statistics

Basic problems in the use of applied mathematical statistics for the modeling of complex systems are considered; the possibility of establishing the uniqueness of a mathematical model of optimal complexity by the group method of data handling (GMDH) is demonstrated. The basic shortcoming of contemporary mathematical statistics is that the models used are too simple because until now in regression analysis only one mean-squared error criterion has been used. To define a mathematical model of optimal complexity GMDH uses not one but two criteria and these two criteria assure a unique solution. The resulting equations are so complex that only the multilayered structure of GMDH allows us to write them down. The method works not only whenK N but also whenK >N(Kis the number of coefficients of the regression equation,N is the number of interpolation points). Increasing the area of optimization raises the accuracy of the model. The second criterion should be heuristic. Mean-squared error defined on a test sequence is used. The division of data into training and test sequences is the basic object of so-called goal-directed regularization. A second shortcoming of contemporary applied mathematical statistics is the absence of freedom of decision in the terminology of D. Gabor. The GMDH selection-type algorithm realizes both the self-organization and freedom of decision criteria. GMDH is a nonparametric procedure and does not require many of the concepts of mathematical statistics.     

Regularized Negative Correlation Learning for Neural Network Ensembles

Negative correlation learning (NCL) is a neural network ensemble learning algorithm that introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean square error (MSE) together with the correlation of the ensemble. This paper analyzes NCL and reveals that the training of NCL (when $lambda=1$) corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This analysis explains the reason why NCL is prone to overfitting the noise in the training set. This paper also demonstrates that tuning the correlation parameter $lambda$ in NCL by cross validation cannot overcome the overfitting problem. The paper analyzes this problem and proposes the regularized negative correlation learning (RNCL) algorithm which incorporates an additional regularization term for the whole ensemble. RNCL decomposes the ensemble's training objectives, including MSE and regularization, into a set of sub-objectives, and each sub-objective is implemented by an individual neural network. In this paper, we also provide a Bayesian interpretation for RNCL and provide an automatic algorithm to optimize regularization parameters based on Bayesian inference. The RNCL formulation is applicable to any nonlinear estimator minimizing the MSE. The experiments on synthetic as well as real-world data sets demonstrate that RNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set.      

Polynomial-Time Decomposition Algorithms for Support Vector Machines

This paper studies the convergence properties of a general class of decomposition algorithms for support vector machines (SVMs). We provide a model algorithm for decomposition, and prove necessary and sufficient conditions for stepwise improvement of this algorithm. We introduce a simple rate certifying condition and prove a polynomial-time bound on the rate of convergence of the model algorithm when it satisfies this condition. Although it is not clear that existing SVM algorithms satisfy this condition, we provide a version of the model algorithm that does. For this algorithm we show that when the slack multiplier C satisfies 1/2 C mL, where m is the number of samples and L is a matrix norm, then it takes no more than 4LC ² m ⁴/ iterations to drive the criterion to within of its optimum.