Real-time fingertip tracking and gesture recognition

Augmented desk interfaces and other virtual reality systems depend on accurate, real-time hand and fingertip tracking for seamless integration between real objects and associated digital information. We introduce a method for discerning fingertip locations in image frames and measuring fingertip trajectories across image frames. We also propose a mechanism for combining direct manipulation and symbolic gestures based on multiple fingertip motions. Our method uses a filtering technique, in addition to detecting fingertips in each image frame, to predict fingertip locations in successive image frames and to examine the correspondences between the predicted locations and detected fingertips. This lets us obtain multiple complex fingertip trajectories in real time and improves fingertip tracking. This method can track multiple fingertips reliably even on a complex background under changing lighting conditions without invasive devices or color markers.     

Agent-based reconfigurable architecture for real-time object tracking

Most conventional object tracking algorithms are implemented on general-purpose processors in software due to its great flexibility. However, the real-time performance is hard to achieve due to the inherent characteristics of the sequential processing of these processors. To tackle this issue, a reconfigurable system-on-chip (rSoC) platform with microprocessors and FPGAs is applied in this paper. To simplify the hardware/software interface, a Belief–Desire–Intention (BDI)-based multi-agent architecture is proposed as the unified framework. Then an agent-based task graph and two heuristic partitioning methods are proposed to partition the hardware and software on an rSoC platform. Compared to the module-based architecture, this BDI-based multi-agent architecture provides more efficiency, flexibility, autonomy, and scalability for the real-time tracking systems. A particle swarm optimization (PSO)-based object detection and tracking algorithm is applied to evaluate the proposed architecture. Extensive experimental results of object tracking demonstrate that the proposed architecture is efficient and highly robust with real-time performance.     

Agent-based framework to individual tracking in unconstrained environments

Agents with intelligent perceptual capabilities are considered state-of-the-art in advanced intelligent systems. In addition, a multiagent system is considered an enabling technology for applications that rely on distributed and parallel processing, including data, information and knowledge in complex computing environments. With the aim of creating advanced intelligent systems with visual perception, this paper presents an agent-based framework to individual tracking in unconstrained environments. The framework has three types of agents that interact using the Contract Net Protocol. The face detector and tracker agents perform fully automatic single-sample face recognition using the Viola-Jones and the Scale Invariant Feature Transform/Speeded Up Robust Features algorithms. The experimental results show that the framework adequately recognizes and tracks individuals in unconstrained environments, indicating the path the individuals have taken and the time they spent in the field of view of the surveillance agents. Some of the open source framework advantages are the distribution in heterogeneous infrastructures, the expansion with new agents using different face recognition algorithms (e.g., Eigenfaces), and the individual tracking logs that can be used in different ways, e.g., improve security in surveillance areas such as automated teller machines, self-paying kiosks, movie box offices, and malls.     

基于范例集实现复杂场景下的动态手势跟踪

范例集作为近年来视觉跟踪器的一种表示方法,已应用于手势跟踪与识别。但是基于范例集的手势跟踪往往不能实现复杂场景下手部轮廓特征的精确提取,并且不能在手部位置与方向任意变化情况下实现手部轮廓的提取与跟踪,极易造成手部动作的不准确预测,从而影响手势跟踪效果与手势识别率。本文提出一种新的基于范例集的跟踪器：CEE(CAMSHIFT Embedded Exemplar)跟踪器,实现复杂场景下的动态手势跟踪。在学习阶段,利用ICAMSHIFT（Improved CAMSHIFT）算法提取手部轮廓特征并生成范例集,同时建立手势的动态HMM模型;在跟踪阶段,首先利用由ICAMSHIFT算法获取的手部特征和HMM概率模型预测手势动作,然后根据学习所得范例集获取当前手部轮廓。实验结果表明,算法能实现复杂场景下的准确手势跟踪,并能在手部位置与方向任意变化情况下实现手部轮廓的提取与跟踪。此外,在严重遮挡情况下也能取得不错的跟踪效果。     

基于计算机视觉的可变形手势跟踪算法改进研究

手势是人们最为常用的、最为自然的表达方式,人机交互的方式迎合了社会的需要。其成本低、适用性强、操作方便,也符合了人们的要求,得到了越来越广泛的关注。传统的手势跟踪算法仅停留在静态的手势跟踪上,当手势变形移动时,会导致跟踪目标从摄像头等设备的视野中消失,导致手势目标消失。本文对传统的CAMSHIFT算法进行了进一步的改进,并对可变形的手势图像的预处理及特征提取做出了研究。有效去除小的零值噪声点和提高了手势跟踪的准确率,这个算法对系统的资源要求不高,并且在仿真实验中有体现出了很好的鲁棒性。仿真实验表明,本文提出的方法有效的解决了一些不确定性问题,能够广泛应用。     

基于压缩感知的实时手势检测和跟踪算法

基于计算机视觉的实时手势检测与跟踪算法是人机交互领域的一项关键技术,传统的手势检测与跟踪算法将检测和跟踪分成两个独立的模块进行,检测与跟踪结果受手势姿态变化、目标遮挡、运动模糊以及外界环境干扰等因素的影响。提出了一种基于压缩感知的实时手势检测和跟踪算法,将基于检测得到的手势信息与基于压缩感知跟踪算法得到的目标信息进行有效融合,从而实现有效的手势检测与跟踪,与传统算法相比,该算法能实现手势跟踪自动初始化和跟踪错误后自我恢复功能。实验结果表明,提出的算法能对手势运动进行快速、连续、准确的识别,满足人机交互的要求。     

Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

Multimedia Tools and Applications - A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve...     

基于Kinect深度图像信息的手势跟踪与识别

针对基于视觉的手势识别技术对环境背景要求较高的问题,提出了一种利用深度信息进行手势提取和识别的研究方案。采用Kinect深度摄像头,通过中值滤波以及深度信息与邻域特点来分割手部区域并用Canny算子提取出手势轮廓,再以深度图像的凸缺陷指尖来完成对指尖的检测,从而实现对数字手势1到5的手势识别。该方法可快速有效地对指尖进行检测,鲁棒性和稳定性都比其他方法更好。实验结果表明,该手势识别方案的平均识别率达到92%,证明了该方法的可行性。     

一种肤色干扰下的变形手势跟踪方法

介绍了一种基于单目视觉的肤色干扰下的变形手势跟踪方法。根据跟踪过程中所用到的基本手势特征,提出了一种基于PGH（成对几何直方图）的静态手势识别方法。为了解决跟踪过程中的肤色干扰问题,实现了基于Kalman滤波器的手势预测跟踪。为了解决跟踪过程中的初始化问题,提出了一种基于层次结构的跟踪初始化解决方案。实验结果表明,该方法能够在肤色干扰的情况下有效地对变形手势进行跟踪,并能够满足基于视觉的实时人机交互的要求。     

Markerless tracking and gesture recognition using polar correlation of camera optical flow

We present a novel, real-time, markerless vision-based tracking system, employing a rigid orthogonal configuration of two pairs of opposing cameras. Our system uses optical flow over sparse features to overcome the limitation of vision-based systems that require markers or a pre-loaded model of the physical environment. We show how opposing cameras enable cancellation of common components of optical flow leading to an efficient tracking algorithm that captures five degrees of freedom including direction of translation and angular velocity. Experiments comparing our device with an electromagnetic tracker show that its average tracking accuracy is 80 % over 185 frames, and it is able to track large range motions even in outdoor settings. We also present how our tracking system can be used for gesture recognition by combining it with a simple linear classifier over a set of 15 gestures. Experimental results show that we are able to achieve 86.7 % gesture recognition accuracy.     

The perceptive workbench: Computer-vision-based gesture tracking,object tracking,and 3D reconstruction for augmented desks

Abstract. The Perceptive Workbench endeavors to create a spontaneous and unimpeded interface between the physical and virtual worlds. Its vision-based methods for interaction constitute an alternative to wired input devices and tethered tracking. Objects are recognized and tracked when placed on the display surface. By using multiple infrared light sources, the object's 3-D shape can be captured and inserted into the virtual interface. This ability permits spontaneity, since either preloaded objects or those objects selected at run-time by the user can become physical icons. Integrated into the same vision-based interface is the ability to identify 3-D hand position, pointing direction, and sweeping arm gestures. Such gestures can enhance selection, manipulation, and navigation tasks. The Perceptive Workbench has been used for a variety of applications, including augmented reality gaming and terrain navigation. This paper focuses on the techniques used in implementing the Perceptive Workbench and the system's performance.     

基于Agent的体系结构

目前的软件开发方法需要开发人员具有较丰富的计算机和程序设计经验,而且需要知道每一个软构件的详细信息。为软件开发提供了一个基于Agent的体系结构,主要包括构件的识别、构件功能的识别和构件集成,结构能自动识别和集成可重用软构件,不需要熟练的程序设计人员或系统开发人员就可以开发软件,因为通过Agent的使用,应用开发是自动的,结果不但具有较高的理论价值,而且具有很大的实践指导意义。     

Agent-based FMS control

Future manufacturing systems will be integrated into the networks of distributed resources, and at the same time, such systems will be capable of processing both knowledge and material. It will probably be required that manufacturing systems be agile, flexible, and fault-tolerant. Petri nets (PN) and object-oriented design (OOD) are used together in order to develop the integrated agent-based FMS control system. The flexible manufacturing system (FMS) consists of machines, workstations, and automated material handling system, distributed buffer storage sites and computer-based supervisory control, all which can be modeled as an agent in OOD with PN. This paper introduces the design of an agent-based FMS control system through PNs and evaluates the performance using timed placed Petri nets (TPPN). In order to do so, the agent control design, FMS structure has been evaluated in detail and the agent definitions have been submitted. The system includes the sharing and distribution of tasks among agents and the mentioned structure has been simulated by TPPN. The simulation procedure has been realized through Petri Net 2.0—MATLAB Demo Program [Mahulea CF, Motcovschi MH, Pastravanu O. Department of Automatic Control Industrial Informatics, Technical University “Gh. Asachi” of Iasi, Blvd., Mangeron 53A, 6600 Iasi, Romania, 〈http://www.ac.tuiasi.ro/pntool,pntool@ac.tuiasi.ro〉, 2004.]. Each case is modeled, and then the agent's machine processing time is considered in this program. As for the evaluation of the study, the system performance is assessed through the waiting time of the parts in queue and the task distributions.     

Agent-based airline scheduling

The development of an airline schedule can be defined as the art of developing system-wide flight patterns that deliver optimum service to the public in terms of quantity as well as quality. The development of the schedule is market driven with maintenance and crew requirements as constraints. This paper deals with an integrated agent-based approach for the airline scheduling problem. A bidding protocol is used to generate a market based schedule. FIFO and genetic algorithms are used to develop a crew schedule. An expert system combined with the Q-learning algorithm assist operational schedulers in resolving operational conflicts such as delays.     

基于多相机的人脸姿态识别

主动形状模型(ASM)算法被用来进行人脸特征点的精确定位,然后在多相机测量的图像中进行特征点的立体匹配,利用双目视觉和相机三维测距技术可以确定人脸特征点的空间三维位置,从而利用这些特征点的相对位置确定出人脸的姿态。实验结果显示,用该方法进行人脸姿态识别能取得比二维识别更高的精确度。     

一种基于Agent的安全策略管理方法

分析了多域网络环境中安全策略管理的特点及局限性,阐述了安全策略管理中引入Agent的重要性,提出了一种基于Agent的安全策略管理框架。描述了各种Agent的功能,实现了多Agent之间的交互和策略传递,并通过安全本体和XML对安全策略进行表示。