Adaptive filtering under minimum information divergence criterion

Traditional filtering theory is always based on optimization of the expected value of a suitably chosen function of error, such as the minimum mean-square error (MMSE) criterion, the minimum error entropy (MEE) criterion, and so on. None of those criteria could capture all the probabilistic information about the error distribution. In this work, we propose a novel approach to shape the probability density function (PDF) of the errors in adaptive filtering. As the PDF contains all the probabilistic information, the proposed approach can be used to obtain the desired variance or entropy, and is expected to be useful in the complex signal processing and learning systems. In our method, the information divergence between the actual errors and the desired errors is chosen as the cost function, which is estimated by kernel approach. Some important properties of the estimated divergence are presented. Also, for the finite impulse response (FIR) filter, a stochastic gradient algorithm is derived. Finally, simulation examples illustrate the effectiveness of this algorithm in adaptive system training. Recommended by Editorial Board member Naira Hovakimyan under the direction of Editor Jae Weon Choi. This work was supported in part by the National Natural Science Foundation of China under grants 50577037 and 60604010. Badong Chen received the B.S. and M.S. degrees in Control Theory and Engineering from Chongqing University, Chongqing, China, in 1997 and 2003, respectively, and the Ph.D. degree in Computer Science and Technology from Tsinghua University, Beijing China, in 2008. He is currently a Postdoctor of the Institute of Manufacturing Engineering, Department of Precision Instruments and Mechanology, Tsinghua University, Beijing, China. His research interests are in signal processing, adaptive control, and information theoretic aspects of control systems. Yu Zhu received the B.S. of Radio Electronics in 1983 at Beijing Normal University, and the M.S. of Computer Applications in 1993, and the Ph.D. of Mechanical Design and Theory in 2001 at China University of Mining & Technology. He is now a Professor of the Institute of Manufacturing Engineering of Department of Precision and Mechanology of Tsinghua University. His current research interests are parallel machanism and theory, two photon micro-fabrication, ultra-precision motion system and motion control. Jinchun Hu received the Ph.D. in Control Science and Engineering from Nanjing University of Science and Technology, Nanjing, China, in 1998. Since then, he has been a postdoctoral researcher in Nanjing University of Aeronautics and Astronautics in 1999 and Tsinghua University in 2002 respectively. His research interests are in flight control, aerial Robot and intelligent control. Dr. Hu is currently an Associate Professor of the Department of Computer Science and Technology of Tsinghua University, Beijing, China. Zengqi Sun received the B.S. degree from the Department of Automatic Control, Tsinghua University, Beijing, China, in 1966 and the Ph.D. degree in Control Engineering from the Chalmas University of Technology, Sweden, in 1981. He is currently a Professor of the Department of Computer Science and Technology, Tsinghua University, Beijing, China. He is the author or coauthor of more than 100 paper and eight books on control and robotics. His research interests include robotics, intelligent control, fuzzy system, neural networks, and evolutionary computation.