Orthogonal linear regression algorithm based on augmented matrix formulation

The problem of n-dimensional orthogonal linear regression is a problem of finding an n-dimensional hyperplane minimising the sum of Euclidean distances between this hyperplane and a given set of m points, where m n. This nonlinear programming problem has been re-cast in an augmented matrix form and solved as a sequence of iteratively re-weighted least square problems. The proposed algorithm is seen as an alternative to the recently published algorithm by Cavalier and Melloy [1].     

一种快速的单层感知器网络学习算法

提出了一种单层感知器网络训练的新算法,证明了对于线性可分问题和线性不可分问题,算法总是在有限步内终止,算法的迭代次数以模式数为上界;而且,在算法终止时,对于线性可分问题,总是能得到正确的权向量解,所以,如果在算法结束时还不能划分所有模式,则说明给定的模式集确是不可线性划分的。     

Support Vector的几何解释及其在联想记忆中的应用

利用闭凸集上的投影解释support vector的几何意义，利用支持超平面讨论线性分类器的设计问题，对线性可分情形，Support vector由一类数据集合闭凸包在另一类数据集合闭凸包上投影的非零系数向量组成，SVM所决定的超平面位于两投影点关于各自数据集合支持超平面的中间，作为应用，文中给出一种设计理想联想记忆前馈神经网络的方法，它是FP算法的一般化。     

基于相似压缩的近似线性SVM

为了解决模式识别中的近似线性可分问题,提出了一种新的近似线性支持向量机(SVM).首先对近似线性分类中的训练集所形成的两类凸壳进行了相似压缩,使压缩后的凸壳线性可分;基于压缩后线性可分的凸壳,再用平分最近点和最大间隔法求出最优的分划超平面.然后再通过求解最大间隔法的对偶问题,得到基于相似压缩的近似线性SVM.最后,从理论和实证分析两个方面,将该方法与线性可分SVM及推广的平分最近点法进行了对比分析,说明了该方法的优越性与合理性.     

一类可分离SAT问题的O(1.890n)精确算法

布尔可满足性问题（SAT）是指对于给定的布尔公式,是否存在一个可满足的真值指派.这是第1个被证明的NP完全问题,一般认为不存在多项式时间算法,除非P=NP.学者们大都研究了子句长度不超过k的SAT问题（k-SAT）,从全局搜索到局部搜索,给出了大量的相对有效算法,包括随机算法和确定算法.目前,最好算法的时间复杂度不超过O（（2-2/k）ⁿ）,当k=3时,最好算法时间复杂度为O（1.308ⁿ）.而对于更一般的与子句长度k无关的SAT问题,很少有文献涉及.引入了一类可分离SAT问题,即3-正则可分离可满足性问题（3-RSSAT）,证明了3-RSSAT是NP完全问题,给出了一般SAT问题3-正则可分离性的O（1.890ⁿ）判定算法.然后,利用矩阵相乘算法的研究成果,给出了3-RSSAT问题的O（1.890ⁿ）精确算法,该算法与子句长度无关.     

Data Mining with Sparse Grids

O (h _n ⁻¹ n ^{d −1}) instead of O(h _n ^−d ) grid points and unknowns are involved. Here d denotes the dimension of the feature space and h _n = 2 ⁻ⁿ gives the mesh size. To be precise, we suggest to use the sparse grid combination technique [42] where the classification problem is discretized and solved on a certain sequence of conventional grids with uniform mesh sizes in each coordinate direction. The sparse grid solution is then obtained from the solutions on these different grids by linear combination. In contrast to other sparse grid techniques, the combination method is simpler to use and can be parallelized in a natural and straightforward way. We describe the sparse grid combination technique for the classification problem in terms of the regularization network approach. We then give implementational details and discuss the complexity of the algorithm. It turns out that the method scales only linearly with the number of instances, i.e. the amount of data to be classified. Finally we report on the quality of the classifier built by our new method. Here we consider standard test problems from the UCI repository and problems with huge synthetical data sets in up to 9 dimensions. It turns out that our new method achieves correctness rates which are competitive to that of the best existing methods. Received April 25, 2001     

Classification of Parkinson's disease using feature weighting method on the basis of fuzzy C-means clustering

This study presents the application of fuzzy c-means (FCM) clustering-based feature weighting (FCMFW) for the detection of Parkinson's disease (PD). In the classification of PD dataset taken from University of California – Irvine machine learning database, practical values of the existing traditional and non-standard measures for distinguishing healthy people from people with PD by detecting dysphonia were applied to the input of FCMFW. The main aims of FCM clustering algorithm are both to transform from a linearly non-separable dataset to a linearly separable one and to increase the distinguishing performance between classes. The weighted PD dataset is presented to k-nearest neighbour (k-NN) classifier system. In the classification of PD, the various k-values in k-NN classifier were used and compared with each other. Also, the effects of k-values in k-NN classifier on the classification of Parkinson disease datasets have been investigated and the best k-value found. The experimental results have demonstrated that the combination of the proposed weighting method called FCMFW and k-NN classifier has obtained very promising results on the classification of PD.     

Maximum Dispersion and Geometric Maximum Weight Cliques

We consider a facility location problem, where the objective is to “disperse” a number of facilities, i.e., select a given number k of locations from a discrete set of n candidates, such that the average distance between selected locations is maximized. In particular, we present algorithmic results for the case where vertices are represented by points in d-dimensional space, and edge weights correspond to rectilinear distances. Problems of this type have been considered before, with the best result being an approximation algorithm with performance ratio 2. For the case where k is fixed, we establish a linear-time algorithm that finds an optimal solution. For the case where k is part of the input, we present a polynomial-time approximation scheme.     

Fast algorithms for the elimination of common subexpressions

Summary We give two algorithms for computing the set of available expressions at entrance to the nodes of a flow graph. The first takes O(mn) isteps on a program flow graph (one in which no node has more than two successors), where n is the number of nodes and m the number of expressions which are ever computed. A modified version of this algorithm requires O(n ²) steps of an extended type, where bit vector operations are regarded as one step. We present another algorithm which works only for reducible flow graphs. It requires O(n log n) extended steps.Work supported by NSF grant GJ-1052. Portions of this paper appeared in the Proceedings of the IEEE 13th Annual Symposium on Switching and Automata Theory.     

Permutation routing in double-loop networks: design and empirical evaluation

A double-loop network is an undirected graph whose nodes are integers 0,1,…,n−1 and each node u is adjacent to four nodes u±h₁(mod>n), u±h₂(mod>n), where 0<h₁<h₂<n/2. There are initially n packets, one at each of the n nodes. The packet at node u is destined to node π(u), where the mapping uπ(u) is a permutation. The aim is to minimize the number of routing steps to route all the packets to their destinations. If ℓ is the tight lower bound for this number, then the best known permutation routing algorithm takes, on average, 1.98ℓ routing steps (and 2ℓ routing steps in the worst-case).Because the worst-case complexity cannot be improved, we design four new static permutation routing algorithms with gradually improved average-case performances, which are 1.37ℓ, 1.35ℓ, 1.18ℓ, and 1.12ℓ. Thus, the best of these algorithms exceeds the optimal routing by at most 12% on average.To support our algorithm design we develop a program which simulates permutation routing in a network according to the given topology, routing model as well as communication pattern and measure several quality criteria. We have tested our algorithms on a large number of double-loop networks and permutations (randomly generated and standard).     

Characterization of the Sonar Signals Benchmark

We study the classification of sonar targets first introduced by Gorman & Sejnowski (1988). We discovered that not only the training set and the test set of this benchmark are both linearly separable, although by different hyperplanes, but that the complete set of patterns, training and test patterns together, is also linearly separable. The distances of the patterns to the separating hyperplane determined by learning with the training set alone, and to the one determined by learning the complete data set, are presented.     

Fast algorithms for long transportation problems

The paper deals with large transportation problems in which the number of destinations (n) is much larger than the number of origins (m), and in which some or many of the paths from origins to destinations may be inadmissible. Using a new approach, with certain auxilary lists, it is proved that the ordinary simplex algorithm (“Most Negative Rule”) can be performed in O(m²+ m log n) computer operations per iteration, as against O(mn) in the usual approach. A search-in-a-row simplex algorithm (“Row Most Negative Rule”), for which the total number of iterations is probably only somewhat larger, is shown to require just O(m + σm log n) operations per iteration, where σ is the density of the cost matrix (i.e. the proportion of admissible paths). For these rigorous results one needs computer storage which is not considerably larger than that required for storing the cost matrix. For smaller memory an efficient algorithm is also proposed. A general tentative rule for die amount of scanning per iteration is introduced and applied. Computer experiments are reported, confirming theoretical estimates.     

Incorporating fuzzy membership functions into the perceptron algorithm

The perceptron algorithm, one of the class of gradient descent techniques, has been widely used in pattern recognition to determine linear decision boundaries. While this algorithm is guaranteed to converge to a separating hyperplane if the data are linearly separable, it exhibits erratic behavior if the data are not linearly separable. Fuzzy set theory is introduced into the perceptron algorithm to produce a ``fuzzy algorithm' which ameliorates the convergence problem in the nonseparable case. It is shown that the fuzzy perceptron, like its crisp counterpart, converges in the separable case. A method of generating membership functions is developed, and experimental results comparing the crisp to the fuzzy perceptron are presented.     

基于改进的SVM线性可分文本分类算法

支持向量机(Support Vector Machines,简称SVM)根据有限的样本信息在对文本分类的精度和学习能力之间,相比其他的文本分类算法寻求了最佳折中,从而获得了较好的推广能力。而SVM是从线性可分情况下的最优分类面发展而来的,因此对于线性可分文本具有更好的分类效果。给出了一种效率较高的线性可分文本的SVM算法,它在训练的时间复杂度上具有明显的改进,从而可以提高训练效率。结果表明:改进后的SVM算法相比以前的算法大大提高了运行效率。     

Alternating optimization to solve penalized regression‐based clustering model

Two previously proposed heuristic algorithms for solving penalized regression‐based clustering model (PRClust) are (a) an algorithm that combines the difference‐of‐convex programming with a coordinate‐wise descent (DC‐CD) algorithm and (b) an algorithm that combines DC with the alternating direction method of multipliers (DC‐ADMM). In this paper, a faster method is proposed for solving PRClust. DC‐CD uses p × n × (n ? 1)/2 slack variables to solve PRClust, where n is the number of data and p is the number of their features. In each iteration of DC‐CD, these slack variable and cluster centres are updated using a second‐order cone programming (SOCP). DC‐ADMM uses p × n × (n ? 1) slack variables. In each iteration of DC‐ADMM, these slack variables and cluster centres are updated using ADMM. In this paper, PRClust is reformulated into an equivalent model to be solved using alternating optimization. Our proposed algorithm needs only n × (n ? 1)/2 slack variables, which is much less than that of DC‐CD and DC‐ADMM and updates them analytically using a simple equation in each iteration of the algorithm. Our proposed algorithm updates only cluster centres using an SOCP. Therefore, our proposed SOCP is much smaller than that of DC‐CD, which is used to update both cluster centres and slack variables. Experimental results on real datasets confirm that our proposed method is faster and much faster than DC‐ADMM and DC‐CD, respectively.     

Acceleration of the back propagation through dynamic adaptation of the learning rate

Standard backpropagation, as with many gradient based optimization methods converges slowly as neural networks training problems become larger and more complex. In this paper, we present a new algorithm, dynamic adaptation of the learning rate to accelerate steepest descent. The underlying idea is to partition the iteration number domain into n intervals and a suitable value for the learning rate is assigned for each respective iteration interval. We present a derivation of the new algorithm and test the algorithm on several classification problems. As compared to standard backpropagation, the convergence rate can be improved immensely with only a minimal increase in the complexity of each iteration.     

Computing euclidean maximum spanning trees

An algorithm is presented for finding a maximum-weight spanning tree of a set ofn points in the Euclidean plane, where the weight of an edge (p _i ,p _j ) equals the Euclidean distance between the pointsp _i andp _j . The algorithm runs inO(n logh) time and requiresO(n) space;h denotes the number of points on the convex hull of the given set. If the points are vertices of a convex polygon (given in order along the boundary), then our algorithm requires only a linear amount of time and space. These bounds are the best possible in the algebraic computation-tree model. We also establish various properties of maximum spanning trees that can be exploited to solve other geometric problems.     

SVM based adaptive learning method for text classification from positive and unlabeled documents

Automatic text classification is one of the most important tools in Information Retrieval. This paper presents a novel text classifier using positive and unlabeled examples. The primary challenge of this problem as compared with the classical text classification problem is that no labeled negative documents are available in the training example set. Firstly, we identify many more reliable negative documents by an improved 1-DNF algorithm with a very low error rate. Secondly, we build a set of classifiers by iteratively applying the SVM algorithm on a training data set, which is augmented during iteration. Thirdly, different from previous PU-oriented text classification works, we adopt the weighted vote of all classifiers generated in the iteration steps to construct the final classifier instead of choosing one of the classifiers as the final classifier. Finally, we discuss an approach to evaluate the weighted vote of all classifiers generated in the iteration steps to construct the final classifier based on PSO (Particle Swarm Optimization), which can discover the best combination of the weights. In addition, we built a focused crawler based on link-contexts guided by different classifiers to evaluate our method. Several comprehensive experiments have been conducted using the Reuters data set and thousands of web pages. Experimental results show that our method increases the performance (F₁-measure) compared with PEBL, and a focused web crawler guided by our PSO-based classifier outperforms other several classifiers both in harvest rate and target recall.     

Fitting a Step Function to a Point Set

We consider the problem of fitting a step function to a set of points. More precisely, given an integer k and a set P of n points in the plane, our goal is to find a step function f with k steps that minimizes the maximum vertical distance between f and all the points in P. We first give an optimal Θ(nlog n) algorithm for the general case. In the special case where the points in P are given in sorted order according to their x-coordinates, we give an optimal Θ(n) time algorithm. Then, we show how to solve the weighted version of this problem in time O(nlog ⁴ n). Finally, we give an O(nh ²log n) algorithm for the case where h outliers are allowed. The running time of all our algorithms is independent of k.     

Large parallel machines can be extremely slow for small problems

We consider concurrent-write PRAMs with a large number of processors of unlimited computational power and an infinite shared memory. Our adversary chooses a small number of our processors and gives them a 0–1 input sequence (each chosen processor gets a bit, and each bit is given to one processor). The chosen processors are required to compute thePARITY of their input, while the others do not take part in the computation. Ifat most q processors are chosen andq 1/2 log logn, then we show that computing PARITY needsq steps in the worst case. On the other hand, there exists an algorithm which computes PARITY inq steps (for anyq <n) in this model, thus our result is sharp. Surprisingly, if our adversary choosesexactly q of our processors, then they can compute PARITY in [q/2] + 2 steps, and in this case we show that it needs at least [q2] steps. Our result implies that large parallel machines which are efficient when only a small number of their processors are active cannot be constructed. On the other hand, a result of Ajtai and Ben-Or [1] shows that if we haven input bits which contain at most polylogn 1's and polynomially many processors which are all allowed to work, then thePARITY can be solved inconstant time.