Error recovery in the assembly of a Self-Organizing Manipulator by using active visual and force sensing

This paper deals with the assembly of aSelf-OrganizingManipulator (SOM) by using the method ofActive Sensing. We set a 6-axis force/torque sensor in the wrist of a manipulator and a CCD camera in the hand of another manipulator. By cooperation of hand and eye, human beings can do a variety of versatile work. The eyes guide the motion of the hand, while the hand moves to make the object easy to see. We try to construct a system working in the same way as a human being. We integrate a vision system, a manipulator, and a force/torque sensor into a hand-eye working system. The scene simplification is based on the controlled motion of camera and manipulator. A method of theAverage Visible Ratio (AVR) is proposed to evaluate the viewpoint of the movable camera. A strategy for planning the assembly is presented. The efficiency of the system is illustrated by experiments.     

Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models

The use of hand gestures offers an alternative to the commonly used human computer interfaces, providing a more intuitive way of navigating among menus and multimedia applications. This paper presents a system for hand gesture recognition devoted to control windows applications. Starting from the images captured by a time-of-flight camera (a camera that produces images with an intensity level inversely proportional to the depth of the objects observed) the system performs hand segmentation as well as a low-level extraction of potentially relevant features which are related to the morphological representation of the hand silhouette. Classification based on these features discriminates between a set of possible static hand postures which results, combined with the estimated motion pattern of the hand, in the recognition of dynamic hand gestures. The whole system works in real-time, allowing practical interaction between user and application.     

A playmate robot system for playing the rock-paper-scissors game with humans

We have developed a playmate robot system for playing the rock-paper-scissors game with humans. The playmate robot recognizes the hand motions of a human using image processing without attaching any additional units to the human. The playmate robot system consists of three parts: a game management part, a hand motion recognition part, and a robot hand control part. The system functions as follows. (1) Before the game is played, the game management part decides on the motion of the robot hand from amongst rock, paper, and scissors. After the game is played, the robot develops a reaction using speech and facial expressions depending on the result of the game. (2) The hand motion recognition part recognizes the hand motion of the human. It does not use any additional units on the human’s body, only a camera on the robot. (3) The robot hand control part shows the motion of the robot hand. A robot hand has four fingers that are controlled independently. We have played the rock-paper-scissors game with this playmate robot.     

一种面向实时交互的变形手势跟踪方法

变形手势跟踪是基于视觉的人机交互研究中的一项重要内容.单摄像头条件下,提出一种新颖的变形手势实时跟踪方法.利用一组2D手势模型替代高维度的3D手模型.首先利用贝叶斯分类器对静态手势进行识别,然后对图像进行手指和指尖定位,通过将图像特征与识别结果进行匹配,实现了跟踪过程的自动初始化.提出将K-means聚类算法与粒子滤波相结合,用于解决多手指跟踪问题中手指互相干扰的问题.跟踪过程中进行跟踪状态检测,实现了自动恢复跟踪及手势模型更新.实验结果表明,该方法可以实现对变形手势快速、准确的连续跟踪,能够满足基于视觉的实时人机交互的要求.     

面向智能家居的挥手识别算法

目的 用手势控制家电是智能家居发展的趋势之一,传统的静态手势识别算法难以适应复杂的居家环境,特别当使用广角相机或环境干扰大时,为此提出一种动态的挥手识别算法,可以对视频序列中的挥手动作做出响应,以达到控制家电的目的。方法 挥手动作具有周期性且频率相对稳定,算法首先调整长滤波器和短滤波器使其检测到视频内周期性运动的区域,然后利用人手识别算法对周期性运动区域进行验证并确认人手。结果 通过与主流的手势识别算法的对比,在复杂环境下,本文算法将成功次数提高了3%,误触发次数降低了44%,响应时间也降低了近0.4 s。结论 实验结果表明,算法能够满足实际应用需求。此外,算法不基于运动目标检测,运算量极低,可以在较高的图像分辨率下实时运行,并能被移植到嵌入式平台下。     

A non-contact method of capturing low-resolution text for OCR

Document recognition is a lively research area with much effort concentrated on optical character recognition. Less attention is paid to locating and extracting text from the general (non-desktop, non-scanner) environment. Such contact-free extraction of text from a general scene has applications in the context of wearable computing, robotic vision, point and click document capture, or as an aid for visually handicapped people. Here, a novel automatic text reading system is introduced using an active camera focused on text regions already located in the scene (using our recent work). Initially, a located region of text is analysed to determine the optimal zoom that would foveate onto it. Then a number of images are captured over the text region to construct a high-resolution mosaic composite of the whole region. This magnified image of the text is suitable for reading by humans or for recognition by OCR, or even for text-to speech synthesis. Although we employed a low resolution camera, we still obtained very good results. ID="A1"Correspondance and offprint requests to: Dr M. Mirmehdi, Department of Computer Science, University of Bristol, Bristol BS8 1UB, UK. Email: majid@cs.bris.ac.uk     

基于傅立叶描述子和HMM的手势识别

针对家庭服务机器人平台中人机交互的问题,提出基于视觉的手势识别作为人与机器人交互的方式,研究利用傅立叶描述子对手势形状进行描述,并结合支持向量机和隐马尔可夫模型分别对静态手势和动态手势进行分类,实现了静态手势和动态手势的识别。该系统基于新型传感器Kinect,在图像分割阶段结合图像深度信息,可以有效的将手势区域提取出来,在一定范围内具有较强的鲁棒性,特征提取阶段基于傅立叶描述子,使手势识别具有旋转、缩放、平移不变性。针对七种常见静态手势和四种动态手势进行测试,平均识别率分别达到98.8%和96.7%,实验结果表明该系统具有较高的准确度。     

Applications of thermal image and extension theory to biometric personal recognition

In this paper, a new scheme is proposed to design a biometric personal recognition system. First, this paper used the thermal image of the hand by using infrared camera to build the sensor module of the recognition system; the extraction features include the length of palmar midpoint to each finger, palmar profile, finger length and finger width. The thermal image presented in this paper was detects infrared energy and converts it into an electronic signal. Then a new recognition method based on the extension is proposed to perform the core of the personal recognition system. The experimental results confirmed that proposed recognition system has a very high recognition rates, therefore, this paper verification using thermal image of the hand to identity recognition was feasible.     

A Real-Time Approach to the Spotting, Representation, and Recognition of Hand Gestures for Human–Computer Interaction

Aiming at the use of hand gestures for human–computer interaction, this paper presents a real-time approach to the spotting, representation, and recognition of hand gestures from a video stream. The approach exploits multiple cues including skin color, hand motion, and shape. Skin color analysis and coarse image motion detection are joined to perform reliable hand gesture spotting. At a higher level, a compact spatiotemporal representation is proposed for modeling appearance changes in image sequences containing hand gestures. The representation is extracted by combining robust parameterized image motion regression and shape features of a segmented hand. For efficient recognition of gestures made at varying rates, a linear resampling technique for eliminating the temporal variation (time normalization) while maintaining the essential information of the original gesture representations is developed. The gesture is then classified according to a training set of gestures. In experiments with a library of 12 gestures, the recognition rate was over 90%. Through the development of a prototype gesture-controlled panoramic map browser, we demonstrate that a vocabulary of predefined hand gestures can be used to interact successfully with applications running on an off-the-shelf personal computer equipped with a home video camera.     

基于深度信息的手势识别算法研究

针对在复杂背景中传统手势识别算法的识别率低问题,利用Kinect的深度摄像头获取深度图像,分割出手势区域后进行预处理;提取手势的几何特征,并提出深度信息的同心圆分布直方图特征,融合手势的几何特征和深度信息的同心圆分布直方图特征;学习训练随机森林分类器进行手势识别.文中通过在复杂背景条件下对常见的“石头”、“剪刀”、“布”3种手势进行测试,实验结果表明:文中所提方法具有很好的平移,旋转和缩放不变性,能适应复杂环境的变化.     

Real-time computation of disparity for hand-pair gesture recognition using a stereo webcam

This paper presents an algorithm for the real-time computation of disparity using video stereo images captured by a stereo webcam. This algorithm is designed to provide both real-time throughput and robust disparity estimation for real-world applications where computation is limited to a pre-defined region-of-interest (ROI). More specifically, this algorithm is used as part of a hand-pair gesture recognition application where the disparity is computed for two ROI around a hand-pair identified by the segmentation component of the recognition application. The developed algorithm provides the required relative difference in disparity with background at high frame rates for the hand-pair gesture recognition application. The results obtained with an inexpensive commercial VGA stereo webcam show a robust disparity computation of 20?ms/frame enabling real-time hand-pair gesture recognition at 25?fps with >90% recognition rate for a maximum hand speed of 40?cm/s and for hand distances between 30 and 150?cm away from the camera.     

3D-freehand-pose initialization based on operator’s cognitive behavioral models

Tracking, recognition and interaction based on 3D freehand are a part of our virtual assembly system, in which monocular camera is used to input online freehand videos and the hand pose tracker requires a reliable initial pose in the first frame. A novel approach to initializing 3D pose and position of freehand is put forward in this paper visualization of 3D hand model and modeling the operators’ cognitive behaviors. Our approach is composed of three phases: hand posture recognition, coarse-tuning and fine-tuning. The operator moves his/her hand onto the to meet the needs of our virtual assembly system. The main contribution of this paper is that the three core techniques are for the first time integrated together, including human–computer interaction (HCI) in the process of initializing, projection of the 3D hand model in the period of coarse-tuning time. Then, the computer repeatedly fine-tunes the 3D hand model until the projection of the 3D hand model is completely superimposed onto the operator’s hand image. We focus on exploring and modeling cognitive behavior of operator’s hand upon which we design our initialization algorithm. Our research shows that cognitive behavioral models are not only beneficial to reducing cognitive loads for operators, because it makes the computers cater for the changes of the operators’ hand poses, but also helpful to address high dimensionality of articulated 3D hand model. Our experimental results also show that the approach presented in this paper is easier, more pleasurable and satisfactory experience for the operators. Our initialization system has successfully been applied to our 3D freehand tracking system and a simulation virtual assembly system.     

基于视觉技术的手势跟踪与动作识别算法

对工业生产线上规程化操作动作进行手势跟踪与动作识别研究。首先选取YCbCr颜色模型进行手部区域识别,获得完整手部区域;然后利用Euclidean距离变换计算相邻2个手部运动轨迹点之间的距离和各帧图像的手部运动速度;再利用扩展有限状态机模型实现手部运动的分割,将分割的多个动作与建立的动作模板匹配,利用Hausdorff距离匹配法判断匹配结果的准确性,实现手部动作的识别。实验结果表明：该手部动作识别算法对背景干扰和摄像头轻微震动具有一定的抗噪能力,有较高的动作识别正确率,能够满足现实工作环境下的应用需求。     

Vision-based single-stroke character recognition for wearable computing

Particularly when compared to traditional tools such as a keyboard or mouse, wearable computing data entry tools offer increased mobility and flexibility. Such tools include touch screens, hand gesture and facial expression recognition, speech recognition, and key systems. We describe a new approach for recognizing characters drawn by hand gestures or by a pointer on a user's forearm captured by a digital camera. We draw each character as a single, isolated stroke using a Graffiti-like alphabet. Our algorithm enables effective and quick character recognition. The resulting character recognition system has potential for application in mobile communication and computing devices such as phones, laptop computers, handheld computers and personal data assistants.     

基于视觉的多特征手势识别

手势是一种自然直观的交互方式,基于视觉的手势识别是实现新一代人机交互的关键技术。本文在已有的手势识别技术基础上,从手势分割及手势表示两方面着手,提出了一种单目视觉下的手势识别方法。利用颜色特征检测肤色区域,成功分割出人手;利用人手的轮廓及凸缺陷检测指尖,再利用指尖的数目和方位来表示一个手势,进而结合轮廓长度和面积等几何特征完成手势识别。传统的指尖检测方法需要遍历并扫描手掌外轮廓,计算量大,本文通过凸缺陷检测指尖,减少了计算量,提高了指尖检测的速度。实验结果表明,本文的方法具有很好的鲁棒性及实时性,能适应环境的变化。     

基于深度信息的实时手势识别和虚拟书写系统

鉴于无接触体感交互技术在人机交互领域的成功应用,提出了一种基于Kinect深度相机的实时隔空虚拟书写方法。结合颜色和深度数据检测和分割出手掌区域;进一步,通过修改的圆扫描转换算法获得手指的个数,以识别不同的手势指令;根据指尖检测从指尖的运动轨迹分割出独立的字符或汉字运动轨迹,并采用随机森林算法识别该字符或汉字。这种基于深度信息的手势检测和虚拟书写方法可以克服光照和肤色重叠的影响,可靠实时地检测和识别手势和隔空书写的文字,其识别率达到93.25%,识别速度达到25 frame/s。     

一种基于嵌入式系统实时交互的手势识别方法*

提出了一种在单摄像头条件下基于嵌入式系统的手势识别方法。通过拟合手势图的外接多边形,找出其所对应的手势缺陷图,并建立手势与手势缺陷图的一一映射,利用手势缺陷图的特征来匹配和识别不同的手势。算法还将手势的跟踪与识别有机地统一起来,通过预测下一帧中手势出现的粗略位置大大降低识别步骤的计算量。该算法在实际应用的嵌入式平台下,能快速、准确地实现手势的识别,能够满足实时人机交互的要求。     

基于6416和FPGA的手部三模态识别装置设计与实现

介绍了将手形、掌纹和手掌静脉身份识别理论与FPGA和DSP数字处理系统相结合构成能够方便用于门禁、考勤等的具有高可靠性和高安全性要求的快速身份识别装置。装置采用可见光和近红外双摄像头提高对手形、掌纹和手掌静脉图像的有效获取。利用FPGA实现双摄像头图像数据的同步采集、数据缓存以及对液晶显示屏、补光系统等的控制,以减少DSP的负担,使DSP能够专注于对手形、掌纹和手掌静脉的数据处理识别工作,保证系统运行的实时性。     

A probabilistic integrated object recognition and tracking framework

This paper describes a probabilistic integrated object recognition and tracking framework called PIORT, together with two specific methods derived from it, which are evaluated experimentally in several test video sequences. The first step in the proposed framework is a static recognition module that provides class probabilities for each pixel of the image from a set of local features. These probabilities are updated dynamically and supplied to a tracking decision module capable of handling full and partial occlusions. The two specific methods presented use RGB color features and differ in the classifier implemented: one is a Bayesian method based on maximum likelihood and the other one is based on a neural network. The experimental results obtained have shown that, on one hand, the neural net based approach performs similarly and sometimes better than the Bayesian approach when they are integrated within the tracking framework. And on the other hand, our PIORT methods have achieved better results when compared to other published tracking methods in video sequences taken with a moving camera and including full and partial occlusions of the tracked object.