A “soft” K-nearest neighbor voting scheme

The K-Nearest Neighbor (K-NN) voting scheme is widely used in problems requiring pattern recognition or classification. In this voting scheme an unknown pattern is classified according to the classifications of its K nearest neighbors. If a majority of the K nearest neighbors have a given classification C*, then the unknown pattern is also given the classification C*. Although the scheme works well it is sensitive to the number of nearest neighbors, K, which is used. In this paper we describe a fuzzy K-NN voting scheme in which effectively the value of K varies automatically according to the local density of known patterns. We find that the new scheme consistently outperforms the traditional K-NN algorithm. © 2001 John Wiley & Sons, Inc.     

A fuzzy vector valued KNN-algorithm for automatic outlier detection

The K nearest neighbors approach is a viable technique in time series analysis when dealing with ill-conditioned and possibly chaotic processes. Such problems are frequently encountered in, e.g., finance and production economics. More often than not, the observed processes are distorted by nonnormal disturbances, incomplete measurements, etc. that undermine the identification, estimation and performance of multivariate techniques. If outliers can be duly recognized, many crisp statistical techniques may perform adequately as such. Geno-mathematical programming provides a connection between statistical time series theory and fuzzy regression models that may be utilized e.g., in the detection of outliers. In this paper we propose a fuzzy distance measure for detecting outliers via geno-mathematical parametrization. Fuzzy KNN is connected as a linkable library to the genetic hybrid algorithm (GHA) of the author, in order to facilitate the determination of the LR-type fuzzy number for automatic outlier detection in time series data. We demonstrate that GHA[Fuzzy KNN] provides a platform for automatically detecting outliers in both simulated and real world data.     

适用于小样本的双邻接图判别分析算法

作为一种常用的降维方法,适用于小样本的监督化拉普拉斯判别分析方法通过使用图嵌入的判别近邻分析得到了很好的降维效果。但该方法在构建近邻图时,在K近邻中寻找同类和异类样本点存在数据不平衡问题;此外,在优化该方法的目标函数时,没有全面考虑到类间信息,从而会在一定程度上降低该方法的性能。针对以上两个问题,本文提出了适用于小样本的双邻接图判别分析方法。首先该方法分别在同类和异类样本中找出K个近邻点,然后使用这K个类内近邻点和K个类间近邻点来构造双邻接图,这样可以确保邻接图中既有同类样本点也有异类样本点,且数目相同。然后该方法在目标函数的推导结果中加入了类间拉普拉斯散度矩阵,从而使优化得到的投影矩阵融入更多的类间信息。在Yale和ORL人脸数据集上进行实验,并与同类方法相比,结果表明本文提出的适用于小样本的双邻接图判别分析方法能够得到更好的降维效果。     

Image Compression and Reconstruction Using pi<Subscript><Emphasis Type="Italic">t</Emphasis></Subscript>-Sigma Neural Networks

A high-order feedforward neural architecture, called pi _t -sigma (π _t σ) neural network, is proposed for lossy digital image compression and reconstruction problems. The π _t σ network architecture is composed of an input layer, a single hidden layer, and an output layer. The hidden layer is composed of classical additive neurons, whereas the output layer is composed of translated multiplicative neurons (π _t -neurons). A two-stage learning algorithm is proposed to adjust the parameters of the π _t σ network: first, a genetic algorithm (GA) is used to avoid premature convergence to poor local minima; in the second stage, a conjugate gradient method is used to fine-tune the solution found by GA. Experiments using the Standard Image Database and infrared satellite images show that the proposed π _t σ network performs better than classical multilayer perceptron, improving the reconstruction precision (measured by the mean squared error) in about 56%, on average.     

Comparison of connectionist and multiple regression approaches for prediction of body weight of goats

The Attappady Black goat is a native goat breed of Kerala in India and is mainly known for its valuable meat and skin. In this work, a comparative study of connectionist network [also known as artificial neural network (ANN)] and multiple regression is made to predict the body weight from body measurements in Attappady Black goats. A multilayer feed forward network with backpropagation of error learning mechanism was used to predict the body weight. Data collected from 824 Attappady Black goats in the age group of 0–12 months consisting of 370 males and 454 females were used for the study. The whole data set was partitioned into two data sets, namely training data set comprising of 75 per cent data (277 and 340 records in males and females, respectively) to build the neural network model and test data set comprising of 25 per cent (93 and 114 records in males and females, respectively) to test the model. Three different morphometric measurements viz. chest girth, body length and height at withers were used as input variables, and body weight was considered as output variable. Multiple regression analysis (MRA) was also done using the same training and testing data sets. The prediction efficiency of both models was compared using the R ² value and root mean square error (RMSE). The correlation coefficients between the actual and predicted body weights in case of ANN were found to be positive and highly significant and ranged from 90.27 to 93.69%. The low value of RMSE and high value of R ² in case of connectionist network (RMSE: male—1.9005, female—1.8434; R ²: male—87.34, female—85.70) in comparison with MRA model (RMSE: male—2.0798, female—2.0836; R ²: male—84.84, female—81.74) show that connectionist network model is a better tool to predict body weight in goats than MRA.     

Prototype sample selection based on minimization of the complete cross validation functional

A method of prototype sample selection from a training set for a classifier of K nearest neighbors (KNN), based on minimization of the complete cross validation functional, is proposed. The optimization leads to reduction of the training set to the minimum sufficient number of prototypes, removal (censoring) of noise samples, and improvement of the generalization ability, simultaneously.     

Approximation Algorithms for Access Network Design

Abstract. We consider the problem of designing a minimum cost access network to carry traffic from a set of endnodes to a core network. Trunks are available in K types reflecting economies of scale . A trunk type with a high initial overhead cost has a low cost per unit bandwidth and a trunk type with a low overhead cost has a high cost per unit bandwidth. We formulate the problem as an integer program. We first use a primal—dual approach to obtain a solution whose cost is within O(K ² ) of optimal. Typically the value of K is small. This is the first combinatorial algorithm with an approximation ratio that is polynomial in K and is independent of the network size and the total traffic to be carried. We also explore linear program rounding techniques and prove a better approximation ratio of O(K) . Both bounds are obtained under weak assumptions on the trunk costs. Our primal—dual algorithm is motivated by the work of Jain and Vazirani on facility location [7]. Our rounding algorithm is motivated by the facility location algorithm of Shmoys et al. [12].     

Continuous K-Nearest Neighbor Query for Moving Objects with Uncertain Velocity

One of the most important queries in spatio-temporal databases that aim at managing moving objects efficiently is the continuous K-nearest neighbor (CKNN) query. A CKNN query is to retrieve the K-nearest neighbors (KNNs) of a moving user at each time instant within a user-given time interval [t _s , t _e ]. In this paper, we investigate how to process a CKNN query efficiently. Different from the previous related works, our work relieves the past assumption, that an object moves with a fixed velocity, by allowing that the velocity of the object can vary within a known range. Due to the introduction of this uncertainty on the velocity of each object, processing a CKNN query becomes much more complicated. We will discuss the complications incurred by this uncertainty and propose a cost-effective P² KNN algorithm to find the objects that could be the KNNs at each time instant within the given query time interval. Besides, a probability-based model is designed to quantify the possibility of each object being one of the KNNs. Comprehensive experiments demonstrate the efficiency and the effectiveness of the proposed approach.

On-line coloring and cliques covering for KK s,t -free graphs

The problem of on-line coloring of an arbitrary graphs is known to be hard. Here we consider the problem of on-line coloring in the simplified situation where the input graph is KK_s,t-free. We show that the on-line coloring algorithm with the First Fit strategy of proposed by Capponi and Pilotto in [1] is the best one for KK_1,t-free graphs (t≥3). A.Capponi and C.Pilotto have shown that this algorithm achieves a competitive ratio of t−1 while we show that it is the best possible. However for the family of KK_s,t-free graphs (s≥2, t≥2) there exists no on-line coloring algorithm with a competitive function. The problem of an on-line cliques covering for these families is hard. There exists no on-line cliques covering algorithm with a competitive function for the family of KK_s,t-free graphs (s≥ 1, t≥3). The additional assumption that the input graph is given in a connected way does not help a lot and does not change our results described above.     

Makespan minimization of multi-slot just-in-time scheduling on single and parallel machines

The paper addresses the problem of multi-slot just-in-time scheduling. Unlike the existing literature on this subject, it studies a more general criterion—the minimization of the schedule makespan rather than the minimization of the number of slots used by schedule. It gives an O(nlog ² n)-time optimization algorithm for the single machine problem. For arbitrary number of m>1 identical parallel machines it presents an O(nlog n)-time optimization algorithm for the case when the processing time of each job does not exceed its due date. For the general case on m>1 machines, it proposes a polynomial time constant factor approximation algorithm.     

使用BP神经网络缓解协同过滤推荐算法的稀疏性问题

推荐质量低是协同过滤推荐技术面临的主要难题之一、数据集的极端稀疏是造成推荐质量低的主要原因之一．常见的降维法和智能Agent法虽然某种程度上能缓解这个问题,但会导致信息损失和适应性等问题．设计了一个新的协同过滤算法,根据用户评分向量交集大小选择候选最近邻居集,采用BP神经网络预测用户对项的评分,减小候选最近邻数据集的稀疏性．该算法避免了降维法和智能Agent法的缺点,而且实验结果表明,该方法能提高预测值的准确度,从而提高协同过滤推荐系统的推荐质量．     

基于Voronoi-R*的隐私保护路网k近邻查询方法

针对已有的保护位置隐私路网k近邻查询依赖可信匿名服务器造成的安全隐患,以及服务器端全局路网索引利用效率低的缺陷,提出基于路网局部索引机制的保护位置隐私路网近邻查询方法.查询客户端通过与LBS服务器的一轮通信获取局部路网信息,生成查询位置所在路段满足l-路段多样性的匿名查询序列,并将匿名查询序列提交LBS服务器,从而避免保护位置隐私查询对可信第三方服务器的依赖.在LBS服务器端,提出基于路网基本单元划分的分段式近邻查询处理策略,对频繁查询请求路网基本单元,构建基于路网泰森多边形和R^*树的局部Vor-R^*索引结构,实现基于索引的快速查找.对非频繁请求路网基本单元,采用常规路网扩张查询处理.有效降低索引存储规模和基于全局索引进行无差异近邻查询的访问代价,在保证查询结果正确的同时,提高了LBS服务器端k近邻查询处理效率.理论分析和实验结果表明,所提方法在兼顾查询准确性的同时,有效地提高了查询处理效率.     

Associative Neural Network

An associative neural network (ASNN) is a combination of an ensemble of the feed-forward neural networks and the K-nearest neighbor technique. The introduced network uses correlation between ensemble responses as a measure of distance among the analyzed cases for the nearest neighbor technique and provides an improved prediction by the bias correction of the neural network ensemble both for function approximation and classification. Actually, the proposed method corrects a bias of a global model for a considered data case by analyzing the biases of its nearest neighbors determined in the space of calculated models. An associative neural network has a memory that can coincide with the training set. If new data become available the network can provide a reasonable approximation of such data without a need to retrain the neural network ensemble. Applications of ASNN for prediction of lipophilicity of chemical compounds and classification of UCI letter and satellite data set are presented. The developed algorithm is available on-line at http://www.virtuallaboratory.org/lab/asnn.     

An optimized instance based learning algorithm for estimation of compressive strength of concrete

This article proposes an optimized instance-based learning approach for prediction of the compressive strength of high performance concrete based on mix data, such as water to binder ratio, water content, super-plasticizer content, fly ash content, etc. The base algorithm used in this study is the k nearest neighbor algorithm, which is an instance-based machine leaning algorithm. Five different models were developed and analyzed to investigate the effects of the number of neighbors, the distance function and the attribute weights on the performance of the models. For each model a modified version of the differential evolution algorithm was used to find the optimal model parameters. Moreover, two different models based on generalized regression neural network and stepwise regressions were also developed. The performances of the models were evaluated using a set of high strength concrete mix data. The results of this study indicate that the optimized models outperform those derived from the standard k nearest neighbor algorithm, and that the proposed models have a better performance in comparison to generalized regression neural network, stepwise regression and modular neural networks models.     

Face recognition with adaptive local hyperplane algorithm

The paper introduces a novel adaptive local hyperplane (ALH) classifier and it shows its superior performance in the face recognition tasks. Four different feature extraction methods (2DPCA, (2D)²PCA, 2DLDA and (2D)²LDA) have been used in combination with five classifiers (K-nearest neighbor (KNN), support vector machine (SVM), nearest feature line (NFL), nearest neighbor line (NNL) and ALH). All the classifiers and feature extraction methods have been applied to the renown benchmarking face databases—the Cambridge ORL database and the Yale database and the ALH classifier with a LDA based extractor outperforms all the other methods on them. The ALH algorithm on these two databases is very promising but more study on larger databases need yet to be done to show all the advantages of the proposed algorithm.     

The Compatibility of Binary Characters on Phylogenetic Networks: Complexity and Parameterized Algorithms

In a recent article, Nakhleh, Ringe and Warnow introduced perfect phylogenetic networks—a model of language evolution where languages do not evolve via clean speciation—and formulated a set of problems for their accurate reconstruction. Their new methodology assumes networks, rather than trees, as the correct model to capture the evolutionary history of natural languages. They proved the NP-hardness of the problem of testing whether a network is a perfect phylogenetic one for characters exhibiting at least three states, leaving open the case of binary characters, and gave a straightforward brute-force parameterized algorithm for the problem of running time O(3 ^k n), where k is the number of bidirectional edges in the network and n is its size. In this paper, we first establish the NP-hardness of the binary case of the problem. Then we provide a more efficient parameterized algorithm for this case running in time O(2 ^k n ²). The presented algorithm is very simple, and utilizes some structural results and elegant operations developed in this paper that can be useful on their own in the design of heuristic algorithms for the problem. The analysis phase of the algorithm is very elegant using amortized techniques to show that the upper bound on the running time of the algorithm is much tighter than the upper bound obtained under a conservative worst-case scenario assumption. Our results bear significant impact on reconstructing evolutionary histories of languages—particularly from phonological and morphological character data, most of which exhibit at most two states (i.e., are binary), as well as on the design and analysis of parameterized algorithms. Research of I.A. Kanj was supported in part by DePaul University Competitive Research Grant.     

APPLICATIONS OF DATA MINING TECHNIQUES TO AUTOMATIC FROG IDENTIFICATION

An intelligent frog call identifier is developed in this work to provide the public with easy online consultation. The raw frog call samples are first filtered by noise removal, high frequency compensation, and discrete wavelet transform techniques in that order. An adaptive end-point detection segmentation algorithm is proposed to effectively separate the individual syllables from the noise. Eight features, including spectral centroid, signal bandwidth, spectral roll-off, threshold-crossing rate, delta spectrum magnitude, spectral flatness, average energy, and mel-frequency cepstral coefficients are extracted and serve as the input parameters of the classifier. Three well-known classifiers, the kth nearest neighboring, a backpropagation neural network, and a naive Bayes classifier, are employed in this work for comparison. A series of experiments were conducted to measure the outcome performance of the proposed work. Experimental results show that the recognition rate of the k-nearest neighbor classifier with the parameters of mel-frequency cepstral coefficients can achieve up to 93.81%. The effectiveness of the proposed frog call identifier is thus verified.     

Efficient Algorithms for Anonymous Byzantine Agreement

This paper considers the Byzantine agreement problem in a completely connected network of anonymous processors. In this network model the processors have no identifiers and can only detect the link through which a message is delivered. We present a polynomial-time agreement algorithm that requires 3⌊(n−t)t/(n−2t)⌋+4 rounds, where n>3t is the number of processors and t is the maximal number of faulty processors that the algorithm can tolerate. We also present an early-stopping variant of the algorithm.     

Acceleration of the back propagation through dynamic adaptation of the learning rate

Standard backpropagation, as with many gradient based optimization methods converges slowly as neural networks training problems become larger and more complex. In this paper, we present a new algorithm, dynamic adaptation of the learning rate to accelerate steepest descent. The underlying idea is to partition the iteration number domain into n intervals and a suitable value for the learning rate is assigned for each respective iteration interval. We present a derivation of the new algorithm and test the algorithm on several classification problems. As compared to standard backpropagation, the convergence rate can be improved immensely with only a minimal increase in the complexity of each iteration.     

Predicting Aflatoxin Contamination in Peanuts: A Genetic Algorithm/Neural Network Approach

Aflatoxin contamination in peanut crops is a problem of significant health and financial importance. Predicting aflatoxin levels prior to crop harvest is useful for minimizing the impact of a contaminated crop and is the goal of our research. Backpropagation neural networks have been used to model problems of this type, however development of networks poses the complex problem of setting values for architectural features and backpropagation parameters. Genetic algorithms have been used in other studies to determine parameters for backpropagation neural networks. This paper describes the development of a genetic algorithm/backpropagation neural network hybrid (GA/BPN) in which a genetic algorithm is used to find architectures and backpropagation parameter values simultaneously for a backpropagation neural network that predicts aflatoxin contamination levels in peanuts based on environmental data. Learning rate, momentum, and number of hidden nodes are the parameters that are set by the genetic algorithm. A three-layer feed-forward network with logistic activation functions is used. Inputs to the network are soil temperature, drought duration, crop age, and accumulated heat units. The project showed that the GA/BPN approach automatically finds highly fit parameter sets for backpropagation neural networks for the aflatoxin problem.