A geometric approach to non-parametric density estimation

A novel non-parametric density estimator is developed based on geometric principles. A penalised centroidal Voronoi tessellation forms the basis of the estimator, which allows the data to self-organise in order to minimise estimate bias and variance. This approach is a marked departure from usual methods based on local averaging, and has the advantage of being naturally adaptive to local sample density (scale-invariance). The estimator does not require the introduction of a plug-in kernel, thus avoiding assumptions of symmetricity and morphology. A numerical experiment is conducted to illustrate the behaviour of the estimator, and it's characteristics are discussed.     

快速核密度估计算法研究进展

作为当前最先进有效的密度估计算法,核密度估计（KDE）得到了广泛的研究。但是其二次的计算复杂度严重阻碍了KDE在具有海量高维数据的实际问题中的应用。为了排除算法计算性能上的障碍,研究者从不同角度提出了多种解决方案。在简要介绍KDE基本算法的基础上,简要分析了近年来提出的一些KDE的快速计算和逼近算法,以便为进一步的研究提供一定的支持与帮助。     

基于自适应核密度估计的运动目标检测

在计算机视频监控系统中,主要的目的是在摄像机固定的视频图像中检测出运动目标,在诸多检测方法中最常用的是减背景技术。减背景技术的关键是背景建模,噪声的干扰、检测方法的自适应性、模型的正确性等问题都是在背景建模过程中必须解决的问题。为了提高建模精度,本文提出了一个非参数化建模技术,称为自适应核密度估计,具有较好的适应性和鲁棒性。它是一种基于场景中像素的概率密度函数来构建的非参数核密度估计的统计模型。     

Probability density estimation using a Gaussian clustering algorithm

A version of the Tråvén's [1] Gaussian clustering algorithm for normal mixture densities is studied. Unlike in the case of the Tråvén's algorithm, no constraints on covariance structure of mixture components are imposed. Simulations suggest that the modified algorithm is a very promising method of estimating arbitrary continuous d-dimensional densities. In particular, the simulations have shown that the algorithm is robust against assuming the initial number of mixture components to be too large.This work was supported in part by the State Committee for Scientific Research (KBN) under grant PB 0589/P3/94/06. It was completed while the second author was on leave to the Department of Statistics, Rice University, Houston, Texas.     

基于核密度估计的K-CFSFDP聚类算法

快速搜索和发现密度峰值的聚类算法(Clustering by Fast Search and Find of Density Peaks,CFSFDP)是一种新的基于密度的聚类算法,它通过发现密度峰值来有效地识别类簇中心,具有聚类速度快、实现简单等优点。针对CFSFDP算法的准确性依赖于数据集的密度估计和截断距离(dc)的人为选择问题,提出一种基于核密度估计的K-CFSFDP算法。该算法利用无参的核密度估计分析数据点的分布特征并自适应地选取dc,从而搜索和发现数据点的密度峰值,并以峰值点数据作为初始聚类中心。基于4个典型数据集的仿真结果表明,K-CFSFDP算法比CFSFDP,K-means和DBSCAN算法具有更高的准确度和更强的鲁棒性。     

Bivariate density estimation using BV regularisation

The problem of bivariate density estimation is studied with the aim of finding the density function with the smallest number of local extreme values which is adequate with the given data. Adequacy is defined via Kuiper metrics. The concept of the taut-string algorithm which provides adequate approximations with a small number of local extrema is generalised for analysing two- and higher dimensional data, using Delaunay triangulation and diffusion filtering. Results are based on equivalence relations in one dimension between the taut-string algorithm and the method of solving the discrete total variation flow equation. The generalisation and some modifications are developed and the performance for density estimation is shown.     

基于自适应密度估计的系统参数辨识

针对噪声分布未知的ARMAX系统,提出了一种自适应非参数噪声密度估计方法,由估计误差动态调整高斯核函数的全局带宽和局部带宽,实现了未知噪声分布密度的自适应估计;通过极小化似然函数,给出了基于噪声密度估计的参数辨识迭代算法,分析了算法的收敛性并给出了算法收敛的充分条件.仿真结果表明本文提出的算法在系统噪声未知时具有较强的抗噪能力和良好的收敛性.     

Control-loop diagnosis using continuous evidence through kernel density estimation

While most previous work in the subject of Bayesian Fault diagnosis and control loop diagnosis use discretized evidence for performing diagnosis (an example of evidence being a monitor reading), discretizing continuous evidence can result in information loss. This paper proposes the use of kernel density estimation, a non-parametric technique for estimating the density functions of continuous random variables. Kernel density estimation requires the selection of a bandwidth parameter, used to specify the degree of smoothing, and a number of bandwidth selection techniques (optimal Gaussian, sample-point adaptive, and smoothed cross-validation) are discussed and compared. Because kernel density estimation is known to have reduced performance in high dimensions, this paper also discusses a number of existing preprocessing methods that can be used to reduce the dimensionality (grouping according to dependence, and independent component analysis). Bandwidth selection and dimensionality reduction techniques are tested on a simulation and an industrial process.     

Variable window adaptive Kernel Principal Component Analysis for nonlinear nonstationary process monitoring

On-line control of nonlinear nonstationary processes using multivariate statistical methods has recently prompt a lot of interest due to its industrial practical importance. Indeed basic process control methods do not allow monitoring of such processes. For this purpose this study proposes a variable window real-time monitoring system based on a fast block adaptive Kernel Principal Component Analysis scheme. While previous adaptive KPCA models allow only handling of one observation at a time, in this study we propose a way to fast update or downdate the KPCA model when a block of data is provided and not only one observation. Using a variable window size procedure to determine the model size and adaptive chart parameters, this model is applied to monitor two simulated benchmark processes. A comparison of performances of the adopted control strategy with various Principal Component Analysis (PCA) control models shows that the derived strategy is robust and yields better detection abilities of disturbances.     

A kernel-based parametric method for conditional density estimation

A conditional density function, which describes the relationship between response and explanatory variables, plays an important role in many analysis problems. In this paper, we propose a new kernel-based parametric method to estimate conditional density. An exponential function is employed to approximate the unknown density, and its parameters are computed from the given explanatory variable via a nonlinear mapping using kernel principal component analysis (KPCA). We develop a new kernel function, which is a variant to polynomial kernels, to be used in KPCA. The proposed method is compared with the Nadaraya-Watson estimator through numerical simulation and practical data. Experimental results show that the proposed method outperforms the Nadaraya-Watson estimator in terms of revised mean integrated squared error (RMISE). Therefore, the proposed method is an effective method for estimating the conditional densities.     

Nonparametric kernel density estimation near the boundary

Standard fixed symmetric kernel-type density estimators are known to encounter problems for positive random variables with a large probability mass close to zero. It is shown that, in such settings, alternatives of asymmetric gamma kernel estimators are superior, but also differ in asymptotic and finite sample performance conditionally on the shape of the density near zero and the exact form of the chosen kernel. Therefore, a refined version of the gamma kernel with an additional tuning parameter adjusted according to the shape of the density close to the boundary is suggested. A data-driven method for the appropriate choice of the modified gamma kernel estimator is also provided. An extensive simulation study compares the performance of this refined estimator to those of standard gamma kernel estimates and standard boundary corrected and adjusted fixed kernels. It is found that the finite sample performance of the proposed new estimator is superior in all settings. Two empirical applications based on high-frequency stock trading volumes and realized volatility forecasts demonstrate the usefulness of the proposed methodology in practice.     

基于Parzen窗法的贝叶斯参数估计

针对贝叶斯估计中所需的非规则概率密度函数, 提出用Parzen窗算法估计相关概率密度, 从而求解不同损失函数下的贝叶斯参数估计器. 实例分析中, 选择一组电阻测量值作为样本, 利用Parzen窗法计算出相应的概率密度函数, 最后用交叉验证法得出了该样本的最小绝对值误差参数估计器.     

A semiparametric density estimation approach to pattern classification

A new multivariate density estimator suitable for pattern classifier design is proposed. The data are first transformed so that the pattern vector components with the most non-Gaussian structure are separated from the Gaussian components. Nonparametric density estimation is then used to capture the non-Gaussian structure of the data while parametric Gaussian conditional density estimation is applied to the rest of the components. Both simulated and real data sets are used to demonstrate the potential usefulness of the proposed approach.     

Multivariate density estimation: A comparative study

This paper is a continuation of the authors' earlier work [1], where a version of the Tråvén's [2] Gaussian clustering neural network (being a recursive counterpart of the EM algorithm) has been investigated. A comparative simulation study of the Gaussian clustering algorithm [1], two versions of plug-in kernel estimators and a version of Friedman's projection pursuit algorithm are presented for two- and three-dimensional data. Simulations show that the projection pursuit algorithm is a good or a very good estimator, provided, however, that the number of projections is suitably chosen. Although practically confined to estimating normal mixtures, the simulations confirm general reliability of plug-in estimators, and show the same property of the Gaussian clustering algorithm. Indeed, the simulations confirm the earlier conjecture that this last estimator proivdes a way of effectively estimating arbitrary and highly structured continuous densities on R^d, at least for small d, either by using this estimator itself or, rather, by using it as a pilot estimator for a newly proposed plug-in estimator.     

Analysing infrequently sampled animal tracking data by incorporating generalized movement trajectories with kernel density estimation

When analysing the movements of an animal, a common task is to generate a continuous probability density surface that characterises the spatial distribution of its locations, termed a home range. Traditional kernel density estimation (KDE), the Brownian Bridges kernel method, and time-geographic density estimation are all commonly used for this purpose, although their applicability in some practical situations is limited. Other studies have argued that KDE is inappropriate analysing moving objects, while the latter two methods are only suitable for tracking data collected at frequent enough intervals such that an object’s movement pattern can be adequately represented using a space–time path created by connecting consecutive points. This research formulates and evaluates KDE using generalised movement trajectories approximated by Delaunay triangulation (KDE-DT) as a method for analysing infrequently sampled animal tracking data. In this approach, a DT is constructed from a point pattern of tracking data in order to approximate the network of movement trajectories for an animal. This network represents the generalised movement patterns of an animal rather than its specific, individual trajectories between locations. Then, kernel density estimates are calculated with distances measured using that network. First, this paper describes the method and then applies it to generate a probability density surface for a Florida panther from radio-tracking data collected three times per week. Second, the performance of the technique is evaluated in the context of delineating wildlife home ranges and core areas from simulated animal locational data. The results of the simulations suggest that KDE-DT produces more accurate home range estimates than traditional KDE, which was evaluated with the same data in a previous study. In addition to animal home range analysis, the technique may be useful for characterising a variety of spatial point patterns generated by objects that move through continuous space, such as pedestrians or ships.     

Approximate inference of the bandwidth in multivariate kernel density estimation

Kernel density estimation is a popular and widely used non-parametric method for data-driven density estimation. Its appeal lies in its simplicity and ease of implementation, as well as its strong asymptotic results regarding its convergence to the true data distribution. However, a major difficulty is the setting of the bandwidth, particularly in high dimensions and with limited amount of data. An approximate Bayesian method is proposed, based on the Expectation-Propagation algorithm with a likelihood obtained from a leave-one-out cross validation approach. The proposed method yields an iterative procedure to approximate the posterior distribution of the inverse bandwidth. The approximate posterior can be used to estimate the model evidence for selecting the structure of the bandwidth and approach online learning. Extensive experimental validation shows that the proposed method is competitive in terms of performance with state-of-the-art plug-in methods.     

Interval estimation for response adaptive clinical trials

In this paper we examine a new method for constructing confidence intervals for the difference of success probabilities to analyze dependent data from response adaptive designs with binary responses. Specifically we investigate the feasibility of the Jeffreys-Perks procedure for interval estimation. Simulation results are derived to demonstrate the performance of the Jeffreys-Perks procedure compared with the profile likelihood method. It is found that both asymptotic methods perform well for small sample sizes despite being approximate procedures.     

Online kernel density estimation for interactive learning

In this paper we propose a Gaussian-kernel-based online kernel density estimation which can be used for applications of online probability density estimation and online learning. Our approach generates a Gaussian mixture model of the observed data and allows online adaptation from positive examples as well as from the negative examples. The adaptation from the negative examples is realized by a novel concept of unlearning in mixture models. Low complexity of the mixtures is maintained through a novel compression algorithm. In contrast to the existing approaches, our approach does not require fine-tuning parameters for a specific application, we do not assume specific forms of the target distributions and temporal constraints are not assumed on the observed data. The strength of the proposed approach is demonstrated with examples of online estimation of complex distributions, an example of unlearning, and with an interactive learning of basic visual concepts.     

Indirect density estimation using the iterative Bayes algorithm

Many practical problems involve density estimation from indirect observations and they are classified as indirect density estimation problems. For example, image deblurring and image reconstruction in emission tomography belong to this class. In this paper we propose an iterative approach to solve these problems. This approach has been successfully applied to emission tomography (Ma, 2008). The popular EM algorithm can also be used for indirect density estimation, but it requires that observations follow Poisson distributions. Our method does not involve such assumptions; rather, it is established simply from the Bayes conditional probability model and is termed the Iterative Bayes (IB) algorithm. Under certain regularity conditions, this algorithm converges to the positively constrained solution minimizing the Kullback-Leibler distance, an asymmetric measure involving both logarithmic and linear scales of dissimilarities between two probability distributions.