Statistical analysis of kernel-based least-squares density-ratio estimation |
| |
Authors: | Takafumi Kanamori Taiji Suzuki Masashi Sugiyama |
| |
Affiliation: | 1.Nagoya University,Nagoya,Japan;2.University of Tokyo,Tokyo,Japan;3.Tokyo Institute of Technology,Tokyo,Japan |
| |
Abstract: | The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), feature selection (mutual information), and conditional probability estimation. Several methods of directly estimating the density ratio have recently been developed, e.g., moment matching estimation, maximum-likelihood density-ratio estimation, and least-squares density-ratio fitting. In this paper, we propose a kernelized variant of the least-squares method for density-ratio estimation, which is called kernel unconstrained least-squares importance fitting (KuLSIF). We investigate its fundamental statistical properties including a non-parametric convergence rate, an analytic-form solution, and a leave-one-out cross-validation score. We further study its relation to other kernel-based density-ratio estimators. In experiments, we numerically compare various kernel-based density-ratio estimation methods, and show that KuLSIF compares favorably with other approaches. |
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |
|