Interactive learning and management of visual information via human-like software robot

To achieve smooth real-world interaction between people and computers, we developed a system that displays a three-dimensional computer-graphic human-like image from the waist up (anthropomorphic software robot: hereinafter “robot”) on the display, that interactively sees and hears, and that has fine and detailed control functions such as facial expressions, line of sight, and pointing at targets with its finger. The robot visually searches and identifies persons and objects in real space that it has learned in advance (registered space, which was our office in this case), manages the history information of the places and times it found objects and/or persons, and tells the user, indicating their three-dimensional positions with line of sight and its finger. It interactively learns new objects and persons with line of with their names and owners. By using this function, the robot can engage in simple dialogue (do a task) with the user. Osamu Hasegawa, Ph.D.: He received the B.E. and M.E. degrees from the Science University of Tokyo, in 1988, 1990 respectively. He received Ph.D. degree from the University of Tokyo, in 1993. Currently, he is a senior research scientist at the Electrotechnical Laboratory (ETL), Tsukuba, Japan. His research interests include Computer Vision and Multi-modal Human Interface. Dr. Hasegawa is a member of the AAAI, the Institute of Electronics, Information and Communication Engineers, Japan (IEICE), Information Processing Society of Japan and others. Katsuhiko Sakaue, Ph.D.: He received the B.E., M.E., and Ph.D. degrees all in electronic engineering from the University of Tokyo, in 1976, 1978 and 1981, respectively. In 1981, he joined the Electrotechnical Laboratory, Ministry of International Trade and Industry, and engaged in researches in image processing and computer vision. He received the Encouragement Prize in 1979 from IEICE, and the Paper Award in 1985 from Information Processing Society of Japan. He is a member of IEICE, IEEE, IPSJ, ITE. Satoru Hayamizu, Ph.D.: He is a leader of Interactive Intermodal Integration Lab. at Electrotechnical Laboratory. He received the B.E., M.E., Ph.D. degrees from Tokyo University. Since 1981, he has been working on speech recognition, spoken dialogue, and communication with artifacts. From 1989 to 1990, he was a visiting scholar in Carnegie Mellon University and in 1994 a visiting scientist in LIMSI/CNRS.