An index and retrieval framework integrating perceptive features and semantics for multimedia databases

Typically, in multimedia databases, there exist two kinds of clues for query: perceptive features and semantic classes. In this paper, we propose a novel framework for multimedia databases index and retrieval integrating the perceptive features and semantic classes to improve the speed and the precision of the content-based multimedia retrieval (CBMR). We develop a semantics supervised clustering based index approach (briefly as SSCI): the entire data set is divided hierarchically into many clusters until the objects within a cluster are not only close in the perceptive feature space but also within the same semantic class, and then an index term is built for each cluster. Especially, the perceptive feature vectors in a cluster are organized adjacently in disk. So the SSCI-based nearest-neighbor (NN) search can be divided into two phases: first, the indexes of all clusters are scanned sequentially to get the candidate clusters with the smallest distances from the query example; second, the original feature vectors within the candidate clusters are visited to get search results. Furthermore, if the results are not satisfied, the SSCI supports an effective relevance feedback (RF) search: users mark the positive and negative samples regarded a cluster as unit instead of a single object; then the Bayesian classifiers on perceptive features and that on semantics are used respectively to adjust retrieval similarity distance. Our experiments show that SSCI-based searching was faster than VA⁺-based searching; the quality of the search result based on SSCI was better than that of the sequential search in terms of semantics; and a few cycles of the RF by the proposed approach can improve the retrieval precision significantly.

Zhiping ShiEmail:

---

Zhiping Shi   received the B.S. degree in engineering at Inner Mongolia University of Technology in Huhhot, China in 1995, the M.S. degree in application of computer science from Inner Mongolia University, China in 2002, and the Ph.D. degree in computer software and theory from Institute of Computing Technology Chinese Academy of Science in 2005. From 1995 to 1999 year, He had been a teacher staff at Inner Mongolia University of Technology. He is an assistant professor at the Key Lab of Intelligent Information Processing of Institute of Computing Technology, Chinese Academy of Science. His research interests include content-based visual information retrieval, image understanding, machine learning and cognitive informatics. Qing He   received his BSc degree from Department of Mathematics, Hebei Normal University in China, and MSc degree from the Department of Mathematics, Zhengzhou University, and the PhD degree from Beijing Normal University in 2000. He has been an Associate Professor of the Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academic of Sciences (KLIIP, ICT, CAS) since 2000. His research interests are in the areas on machine learning, data mining artificial intelligence, neural computing, and cognitive science. Zhongzhi Shi   is a Professor at the Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China. research interests include intelligence science, multiagent systems, and semantic web. He has published 10 books, edited 11 books, and has more than 300 technical papers. His most recent books are Intelligent Agent and Applications and Knowledge Discovery (in Chinese). Mr. Shi is a member of the AAAI. He is the Chair of WG 12.3 of IFIP. He also serves as Vice President of the Chinese Association for Artificial Intelligence. He received the 2nd Grade National Award of Science and Technology Progress in 2002. In 1998 and 2001 he received the 2nd Grade Award of Science and Technology Progress from the Chinese Academy of Sciences.