Multimodal identity tracking in a smart room

The automatic detection, tracking, and identification of multiple people in intelligent environments are important building blocks on which smart interaction systems can be designed. Those could be, e.g., gesture recognizers, head pose estimators or far-field speech recognizers and dialog systems. In this paper, we present a system which is capable of tracking multiple people in a smart room environment while inferring their identities in a completely automatic and unobtrusive way. It relies on a set of fixed and active cameras to track the users and get close-ups of their faces for identification, and on several microphone arrays to determine active speakers and steer the attention of the system. Information coming asynchronously from several sources, such as position updates from audio or visual trackers and identification events from identification modules, is fused at higher level to gradually refine the room’s situation model. The system has been trained on a small set of users and showed good performance at acquiring and keeping their identities in a smart room environment. An erratum to this article can be found at     

A Calculus and logic of resources and processes

Recent advances in logics for reasoning about resources provide a new approach to compositional reasoning in interacting systems. We present a calculus of resources and processes, based on a development of Milner’s synchronous calculus of communication systems, SCCS, that uses an explicit model of resource. Our calculus models the co-evolution of resources and processes with synchronization constrained by the availability of resources. We provide a logical characterization, analogous to Hennessy–Milner logic’s characterization of bisimulation in CCS, of bisimulation between resource processes which is compositional in the concurrent and local structure of systems. An erratum to this article can be found at     

Personal orchestra: a real-time audio/video system for interactive conducting

We present the first multimedia system to conduct a realistic electronic orchestra. Users can control tempo, dynamics, and instrument emphasis of the orchestra through natural conducting gestures with an infrared baton. Using gesture recognition and tempo adjustment algorithms, the system plays back an audio and video recording of an actual orchestra that follows the users conducting in real time. A major achievement of this system is its ability to vary playback speed in real time while avoiding audio artifacts such as pitch changes. The system has been deployed as an exhibit and has become a major attraction of a large Vienna-based music exhibition center.An erratum to this article can be found at     

Biomimetic Representation with Genetic Programming Enzyme

The standard parse tree representation of genetic programming, while a good choice from a generative viewpoint, does not capture the variational demands of evolution. This paper addresses the issue of whether representations in genetic programming might be improved by mimicry of biological behaviors, particularly those thought to be important in the evolution of metabolic pathways, the ‘computational’ structures of the cell. This issue is broached through a presentation of enzyme genetic programming, a form of genetic programming which uses a biomimetic representation. Evaluation upon problems in combinational logic design does not show any significant performance advantage over other approaches, though does demonstrate a number of interesting behaviors including the preclusion of bloat. An erratum to this article can be found at     

3D content-based search using sketches

This paper presents a novel interactive framework for 3D content-based search and retrieval using as query model an object that is dynamically sketched by the user. In particular, two approaches are presented for generating the query model. The first approach uses 2D sketching and symbolic representation of the resulting gestures. The second utilizes non-linear least squares minimization to model the 3D point cloud that is generated by the 3D tracking of the user’s hands, using superquadrics. In the context of the proposed framework, three interfaces were integrated to the sketch-based 3D search system including (a) an unobtrusive interface that utilizes pointing gesture recognition to allow the user manipulate objects in 3D, (b) a haptic–VR interface composed by 3D data gloves and a force feedback device, and (c) a simple air–mouse. These interfaces were tested and comparative results were extracted according to usability and efficiency criteria.     

Object motion tracking using a moving direction estimate and color updates

This paper presents a direction detection and tracking object color update algorithm used to track moving objects that change colors. Different from traditional color-based tracking methods, which use an initial color distribution in order to track objects as long as the object carries the full or partial initial color, this method introduces a color update method used to quickly find the new object color in a new location if the object changes its color partially or completely; the updated color is then used to locate the object. In our algorithm, an initial color pattern is used to track an object using the color. During the tracking, an object’s new location is at first estimated and then used to detect any color change. If the color has changed, a new color pattern is updated based on the changes in the previous color distribution, and then the new color pattern is used to calculate the current location of the object. This algorithm utilizes the property that the movement of an object can be estimated either by using the object’s shadow or by background subtraction. The implementation of our algorithm results in an effective real-time object tracking. The validity of the approach is illustrated by the presentation of experiment results obtained using the methods described in this paper.     

An interactive example-driven approach to graphics recognition in engineering drawings

An interactive example-driven approach to graphics recognition in engineering drawings is proposed. The scenario is that the user first interactively provides an example of a graphic object; the system instantly learns its graphical knowledge and uses the acquired knowledge to recognize the same type of graphic objects. The proposed approach represents the graphical knowledge of an object in terms of its structural components and their syntactical relationships. We summarize four types of geometric constraints for knowledge representation, based on which we develop an algorithm for knowledge acquisition. Another algorithm for graphics recognition using the acquired graphical knowledge is also proposed, which is actually a sequential examination of these constraints. In the algorithm, we first guess the next component’s attributes (e.g., size, position and orientation) by reasoning from an earlier found component and the constraint between them, and then search for this hypothetical component in the drawing. If all of the hypothetical components are found, a graphic object of this type is recognized. For improving the system’s recognition accuracy, we develop a user feedback scheme, which can update the graphical knowledge from both positive (missing) and negative (mis-recognized) examples provided by the user for subsequent recognition. Experiments have shown that our proposed approach is both efficient and effective for recognizing various types of graphic objects in engineering drawings. This paper is an extension of our papers published in ICDAR2003 and GREC2003.     

A color-based particle filter for multiple object tracking in an outdoor environment

Tracking multiple objects is more challenging than tracking a single object. Some problems arise in multiple-object tracking that do not exist in single-object tracking, such as object occlusion, the appearance of a new object and the disappearance of an existing object, updating the occluded object, etc. In this article, we present an approach to handling multiple-object tracking in the presence of occlusions, background clutter, and changing appearance. The occlusion is handled by considering the predicted trajectories of the objects based on a dynamic model and likelihood measures. We also propose target-model-update conditions, ensuring the proper tracking of multiple objects. The proposed method is implemented in a probabilistic framework such as a particle filter in conjunction with a color feature. The particle filter has proven very successful for nonlinear and non-Gaussian estimation problems. It approximates a posterior probability density of the state, such as the object’s position, by using samples or particles, where each state is denoted as the hypothetical state of the tracked object and its weight. The observation likelihood of the objects is modeled based on a color histogram. The sample weight is measured based on the Bhattacharya coefficient, which measures the similarity between each sample’s histogram and a specified target model. The algorithm can successfully track multiple objects in the presence of occlusion and noise. Experimental results show the effectiveness of our method in tracking multiple objects.     

基于瑞萨单片机的四旋翼自主飞行器设计

该四旋翼自主飞行器以瑞萨公司的R5F100LEA单片机为控制核心,包括飞行姿态处理模块,超声波测距模块,红外传感器循迹模块,电机驱动模块以及微处理器模块等.飞行姿态处理由MPU6050加速度计陀螺仪提供,保证飞行器平稳飞行.超声波测距模块和红外传感器循迹模块为飞行器提供导航参数使飞行器可以按照规定航线并以一定高度飞行.本设计中应用了PPM控制方法,PID算法,平滑滤波等,使飞行器实现在一定区域内一键式起飞,稳定飞行,精确降落.并且可以拾取物件,完成空投任务,最终精确降落并停机.     

Specificity enhancement in classification of breast MRI lesion based on multi-classifier

In the conventional CMAC-based adaptive controller design, a switching compensator is designed to guarantee system stability in the Lyapunov stability sense but the undesirable chattering phenomenon occurs. This paper proposes a CMAC-based smooth adaptive neural control (CSANC) system that is composed of a neural controller and a saturation compensator. The neural controller uses a CMAC neural network to online mimic an ideal controller and the saturation compensator is designed to dispel the approximation error between the ideal controller and neural controller without any chattering phenomena. The parameter adaptive algorithms of the CSANC system are derived in the sense of Lyapunov stability, so the system stability can be guaranteed. Finally, the proposed CSANC system is applied to a Chua’s chaotic circuit and a DC motor driver. Simulation and experimental results show the CSANC system can achieve a favorable tracking performance. It should be emphasized that the development of the proposed CSANC system doesn’t need the knowledge of the system dynamics.     

A user-adaptive city guide system with an unobtrusive navigation interface

In this paper, we describe an intelligent location-aware city guide system, which adapts to each user’s preferences, and uses an intuitive “metal detector” interface for navigation. Our system analyzes each user’s past location data history to estimate individual preferences, and allows users to find shops that match their tastes in the same way a metal detector would be used to detect metal objects. The procedure with which the system picks out shops that match each user’s preferences includes a newly developed place learning algorithm, which can efficiently find frequented places, complete with their proper names (e.g. “The Ueno Royal Museum”). We have conducted a series of evaluation tests at a popular shopping district inside Tokyo, and the results validate the effectiveness of our overall approach.     

The Gesture Disagreement Problem in Free-hand Gesture Interaction

Accurately understanding a user’s intention is often essential to the success of any interactive system. An information retrieval system, for example, should address the vocabulary problem (Furnas et al., 1987) to accommodate different query terms users may choose. A system that supports natural user interaction (e.g., full-body game and immersive virtual reality) must recognize gestures that are chosen by users for an action. This article reports an experimental study on the gesture choice for tasks in three application domains. We found that the chance for users to produce the same gesture for a given task is below 0.355 on average, and offering a set of gesture candidates can improve the agreement score. We discuss the characteristics of those tasks that exhibit the gesture disagreement problem and those tasks that do not. Based on our findings, we propose some design guidelines for free-hand gesture-based interfaces.     

Business cycle and corporate failure in France: Is there a link?

The aim of this paper is to extract cyclical factors from companies’ data possibly used in the construction of the scores by the Bank de France and from these scores themselves. The constraints are those of a database including a large number of variables and companies but over a relatively short period. The method chosen is the “principal components method” adapted by Bai and Ng (2000, 2001) in the context of large N and limited T. Later, these factors may be used to take into account the business cycle in the application of scores. An erratum to this article can be found at     

Individual and Social Learning

We use adaptive models to understand the dynamics that lead to efficient and fair outcomes in a repeated Battle of the Sexes game. Human subjects appear to easily recognize the possibility of a coordinated alternation of actions as a mean to generate an efficient and fair outcome. Yet such typical learning models as Fictitious Play and Reinforcement Learning have found it hard to replicate this particular result. We develop a model that not only uses individual learning but also the “social learning” that operates through evolutionary selection. We find that the efficient and fair outcome emerges relatively quickly in our model. JEL classification: B52, D83 An erratum to this article can be found at     

基于惯性传感技术的裁判员手势跟踪研究

针对快速捕捉赛场中裁判员手势判决跟踪问题,依靠图像的动态手势跟踪虚拟现实交互技术,研制了一种基于惯性传感技术的裁判员手势控制器,并进行了相关实验验证。该手势控制器由安装在手部、小臂和大臂的13个传感器节点组成,可用于计算机快速捕捉到裁判员对赛事的判罚,进而与虚拟空间中的物体进行交互。实验结果表明,该手势控制器能够准确地跟踪裁判员手姿势,进而控制虚拟手完成虚拟各类手势等交互操作。     

Elements of the information technology of cytological specimen analysis: Taxonomy and factor analysis

The automated software system “Black Square,” Version 1.2 is described. The system is intended for the automation of image processing, analysis, and recognition. It is an open system for generating new knowledge: objects, algorithms of image processing, recognition procedures originally not intended for image processing, and methods for solving applied problems. The system combines the features of information retrieval, reference, training, and computing systems. This work was partially supported by the Russian Foundation for Basic Research, project nos. 03-07-90406, 05-04-49846, and 05-07-08000; by the INTAS grant no. 04-77-7067; by the Cooperative grant “Image Analysis and Synthesis: Theoretical Foundations and Prototypical Applications in Medical Imaging” within agreement between Italian National Research Council and Russian Academy of Sciences (RAS); by the grant of the RAS in the framework of the Program “Fundamental Science to Medicine.” An erratum to this article is available at .     

Multi-Object Tracking in Video

This paper reports on tracking of multiple objects using color histogram backprojection and motion cues. Four tasks which facilitate this are discussed. The first is an adaptive color histogram backprojection (which builds upon the works of Swain and Ballard) and its application to tracking of multiple objects in video sequences. The second task is designing efficient fast blob detectors for selecting regions of interest in video sequences. The third is motion detection based on color histogram backprojection. Achieving these tasks led to multi-objects tracking. Various video sequences were used to demonstrate effective tracking of multiple objects. Notably, we created an interactive multiple objects tracker (CLICK-IT) which in its present form is set at three objects but can be extended easily. CLICK-IT (CSIRO Laboratory for Imaging by Content and Knowledge—Interactive Television) is a PC-based system which provides the user with an intelligent highlighter pen for sports action replay. It is intended as a truly interactive improvement on the drawing pad technology currently used for video annotation in sports broadcasting. The system uses computer vision techniques to focus attention and track particular objects (player(s), ball, horse(s), …) and semi-automatically annotate the dynamic scene. This paper describes the system including the user interface, the tracking technology based on color and motion information, and system performance evaluation in applications to surveillance-like sequences, running, rugby league football, basketball and soccer. Finally, video scene detection based on color histogram is discussed.     

Connecting the real world with the digital overlay with smart ambient media—applying Peirce’s categories in the context of ambient media

This article is based on the keynote held at the workshop on Events in Multimedia (EiMM09) that took place in conjunction with the ACM Multimedia conference in Beijing in October 2009. The idea of the keynote was to develop and explain the idea of ambient media in general, it is principles, and additionally relate ambient media to information theoretical approaches such as Peirce’s categories, to the theories of decision making, and to theories discussing how smart objects can be made ‘smart’ by simulating the human mind. This article rounds up with practical examples underlining the presented ideas and theories.     

基于特征空间切分建模的变形手势跟踪算法

为解决人机交互中手势形变和无规律运动带来的跟踪难题,提出了一种基于特征空间切分建模的非参数核密度估计算法来实现手势跟踪.首先,在检测模块中利用AdaBoost分类器检测图像中手势的存在,将检测到的手势位置信息传送给跟踪模块,该模块精确提取手势目标从而对其颜色建模.然后,利用目标的颜色模型对各帧图像进行后验概率密度估算,获取运动目标的概率密度图像,将其分解成手势运动区和同色干扰区.最后,对同色干扰区采用混合高斯建模来削弱同色目标的干扰.当目标丢失时启动再检测模块,并利用贝叶斯分类器与方差分类器实现手势目标重检.实验结果表明,该算法通过对特征空间切分建模以及不同分类器的级联解决了变形手势跟踪的同色干扰与再检测难题.该算法提高了跟踪的准确率（>81.5%）,适合于非刚性物体做无规则运动的复杂场景.     

A blackboard approach towards integrated Farsi OCR system

An integrated OCR system for Farsi text is proposed. The system uses information from several knowledge sources (KSs) and manages them in a blackboard approach. Some KSs like classifiers are acquired a priori through an offline training process while others like statistical features are extracted online while recognizing. An arbiter controls the interactions between the solution blackboard and KSs. The system has been tested on 20 real-life scanned documents with ten popular Farsi fonts and a recognition rate of 97.05% in word level and 99.03% in character level has been achieved. An erratum to this article can be found at