Evidential fusion of sensor data for activity recognition in smart homes

Advances in technology have provided the ability to equip the home environment with a layer of technology to provide a truly ‘Smart Home’. These homes offer improved living conditions and levels of independence for the population who require support with both physical and cognitive functions. At the core of the Smart Home is a collection of sensing technology which is used to monitor the behaviour of the inhabitant and their interactions with the environment. A variety of different sensors measuring light, sound, contact and motion provide sufficient multi-dimensional information about the inhabitant to support the inference of activity determination. A problem which impinges upon the success of any information analysis is the fact that sensors may not always provide reliable information due to either faults, operational tolerance levels or corrupted data. In this paper we address the fusion process of contextual information derived from uncertain sensor data. Based on a series of information handling techniques, most notably the Dempster–Shafer theory of evidence and the Equally Weighted Sum operator, evidential contextual information is represented, analysed and merged to achieve a consensus in automatically inferring activities of daily living for inhabitants in Smart Homes. Within the paper we introduce the framework within which uncertainty can be managed and demonstrate the effects that the number of sensors in conjunction with the reliability level of each sensor can have on the overall decision making process.     

基于Bag of Features的手势识别

基于视觉的手势识别中,手势的识别效果易受手势旋转,光照亮度的影响,针对该问题,借鉴了目标识别和图像检索领域的Bag of Features(特征袋)算法,将Bag of Features算法应用到手势识别领域.通过SURF(加速鲁棒性特征)算法提取手势图像的特征描述符,使手势对尺度、旋转、光照具有很强的适应力,再应用Bag of Features算法把SURF特征描述符映射到一个统一维度的向量,即Bag of Features特征向量,再用支持向量机对图像得到的特征向量进行训练分类.实验结果表示,该方法不仅具有较高的时间效率,满足手势识别的实时性,而且即使在很大角度的旋转以及亮度的变化下,仍能达到较高的识别率.     

基于Kinect深度图像信息的手势跟踪与识别

针对基于视觉的手势识别技术对环境背景要求较高的问题,提出了一种利用深度信息进行手势提取和识别的研究方案。采用Kinect深度摄像头,通过中值滤波以及深度信息与邻域特点来分割手部区域并用Canny算子提取出手势轮廓,再以深度图像的凸缺陷指尖来完成对指尖的检测,从而实现对数字手势1到5的手势识别。该方法可快速有效地对指尖进行检测,鲁棒性和稳定性都比其他方法更好。实验结果表明,该手势识别方案的平均识别率达到92%,证明了该方法的可行性。     

Hand gesture recognition using topological features

Multimedia Tools and Applications - Hand Gestures Recognition (HGR) is one of the main areas of research for Human Computer Interaction applications. Most existing approaches are based on local or...     

基于Kinect骨架信息的交通警察手势识别

针对现有的手势识别算法识别率低、鲁棒性弱的问题,提出一种基于Kinect骨架信息的交通警察手势识别方法。从Kinect深度图像中预测人体骨架节点的坐标位置,将节点的运动轨迹作为训练和测试的特征,结合距离加权动态时间规整算法和K-最近邻分类器进行识别。实验表明,在参数最优的情况下,该方法对八种交通警察手势的平均识别率达到98.5%,可应用于智能交通等领域。     

Extraction of spatial-temporal features for vision-based gesture recognition

One of the key problems in a vision-based gesture recognition system is the extraction of spatial-temporal features of gesturing.In this paper an approach of motion-based segmentation is proposed to realize this task.The direct method cooperated with the robust M-estimator to estimate the affine parameters of gesturing motion is used.and based on the dominant motion model the gesturing region is extracted,i.e.,the dominant object.So the spatial-temporal features of gestrues can be extracted.Finally,the dynamic time warping(DTW) method is directly used to perform matching of 12 control gestures(6 for “translation“ orders,6 for “rotation“orders).A small demonstration system has been set up to verify the method,in which a panorama image viewer can be controlled(set by mosaicing a sequence of standard“Garden“ images)with recognized gestures instead of the 3-D mouse tool.     

基于多传感器融合的动态手势识别研究分析

研究利用三类传感器（表面肌电仪、陀螺仪和加速度计）信号的特点进行信息融合,提高可识别动态手势动作的种类和准确率。将动态手势动作分解为手形、手势朝向和运动轨迹三个要素,分别使用表面肌电信号（sEMG）、陀螺仪信号（GYRO）和加速度信号（ACC）进行表征,利用多流HMMs进行动态手势动作的模式识别。对包含有5个运动轨迹和6个静态手形的识别实验结果表明,该方法可以有效地从连续信号中识别动态手势,三类传感器组合使用获得的全局平均识别率达到92％以上,明显高于任意两个传感器组合和仅采用单个传感器获得的平均识别率。实验表明该方法是一种有效的动态手势识别方法,并且相较于传统的动态手势识别的方法更具有优势。     

基于肤色检测与卷积神经网络的手势识别

针对光照变化、背景噪声等复杂环境对手势识别的影响,提出了一种基于YCb Cr空间肤色分割去除背景结合卷积神经网络进行手势识别方法。首先根据人体肤色在YCb Cr颜色空间中的聚类效果,采用基于椭圆模型的肤色检测方法进行手势分割;然后对分割后的手势图像提取骨架与边缘相融合的手势特征图;再通过深层次的Alex Net卷积神经网络结构,对经过融合的手势特征图进行识别。实验结果表明,针对复杂的背景环境,该算法具有较强的鲁棒性,在不同数据集下对手势的平均识别率提升了4%,可以达到99.93%。     

Gabor特征融合的嵌入式人脸识别系统的研究

在嵌入式人脸识别系统中,由于多尺度Gabor抽取特征的维数和数据量过大,不适合在ARM板上实现,提出了多尺度Gabor特征加权融合的方法,很好地解决了图像维数和数据量过大的难点。加权融合过程包括多尺度Gabor特征的提取、特征权值的计算和加权融合过程。同时使用了类Haar特征提取人脸、2DPCA对人脸图像进行降维。基于EELiod 270嵌入式开发平台实现了一个嵌入式系统,结合典型图片库和实际图片进行了人脸识别测试,实践结果表明,系统在保证一定的识别率的同时,大幅度降低了运行时间,实时识别效果良好。     

基于Kinect的动态手势识别研究

针对人机交互领域中基于视觉的传统动态手势识别方法准确率不高、易受不同强度光照影响等问题,对动态手势识别方法进行了研究;首先利用Kinect传感器采集的深度图像对手势进行分割,并基于矩和链码进行手势质心与手势轨迹特征的计算,再利用动态时间规整算法进行手势轨迹特征识别,最后将识别结果传输给六足机器人进行人机交互实验,实现了动态手势对六足机器人的控制;实验结果证明:该方法识别准确率最高可达97%,且不易受光照影响,具有较强的鲁棒性,同时也满足了人机交互需求。     

基于多特征融合的医学图像识别研究*

以医学图像为研究对象,针对任何一类特征都不能很好地表达医学图像的缺点以及进一步提高医学图像的识别率,提出了一种基于特征级数据融合与决策级数据融合相结合的分类方法。实验结果表明,采用特征级数据融合,融合后的特征可以较好地表达医学图像,且减少了后期分类的计算量;采用决策级数据融合,取得了比单个分类器更高的识别率。     

Non-trajectory-based gesture recognition in human-computer interaction based on hand skeleton data

Multimedia Tools and Applications - Currently, no efficient, accurate and flexible gesture recognition algorithm has been developed to recognize non-trajectory-based gesture recognition. Therefore,...     

Similar hand gesture recognition by automatically extracting distinctive features

International Journal of Control, Automation and Systems - With the flourish development of computer vision technology, hand gesture recognition plays a more and more vital role in human-computer...     

用于手势识别的时空融合网络以及虚拟签名系统

由于新型冠状病毒的流行,非接触式个人签名可以在一定程度上降低感染的风险,其将在人们日常的生活中发挥重要作用。因此,提出了一种简单而有效的时空融合网络来实现基于骨架的动态手势识别,并以此为基础开发了一款虚拟签名系统。时空融合网络主要由基于注意力机制的时空融合模块构成,其核心思想是以增量的方式同步实现时空特征的提取与融合。该网络采用不同编码的时空特征作为输入,并在实际应用中采用双滑动窗口机制来进行后处理,从而确保结果更加的稳定与鲁棒。在2个基准数据集上的大量对比实验表明,该方法优于最先进的单流网络方法。另外,虚拟签名系统在一个普通的RGB相机下表现优异,不仅大大降低了交互系统的复杂性,还提供了一种更为便捷、安全的个人签名方式。     

基于表观特征分析的手势识别及其应用

针对复杂背景下的手势识别容易受到环境干扰造成的识别困难问题,通过分析手势的表观特征,提出并实现了一种可用于自然人机交互的手势识别算法。该算法基于Kinect深度图像实现手势区域分割,然后提取手势手指弧度、指间弧度、手指数目等具有旋转缩放不变性的表观特征,运用最小距离法实现快速分类。并将该算法成功运用于实验室三指灵巧手平台,达到了理想的控制效果。实验表明该算法具有良好的鲁棒性,针对九种常用手势,平均识别率达到94.3%。     

基于特征包支持向量机的手势识别

针对类肤色信息或复杂背景的影响,难以通过手势分割得到精确手势轮廓而对后期手势识别率与实时交互的影响,提出了一种基于特征包支持向量机(BOF-SVM)的手势识别方法。采用SIFT算法提取手势图像局部不变性特征点,将手势局部特征向量(尺度不变特征变换(SIFT)描述子)进行K-means聚类生成视觉码书,并通过视觉码书量化每一幅手势图像的视觉码字集合,以此获得手势图像的固定维数的表征向量来训练支持向量机(SVM)多类分类器。该方法只需框定手势所在区域,无需精确地分割人手。实验表明,该方法对9种交互手势的平均识别率达到92.1%,并具有很好的鲁棒性及实时性,能适应环境的变化。     

基于深度信息的手势识别算法研究

针对在复杂背景中传统手势识别算法的识别率低问题,利用Kinect的深度摄像头获取深度图像,分割出手势区域后进行预处理;提取手势的几何特征,并提出深度信息的同心圆分布直方图特征,融合手势的几何特征和深度信息的同心圆分布直方图特征;学习训练随机森林分类器进行手势识别.文中通过在复杂背景条件下对常见的“石头”、“剪刀”、“布”3种手势进行测试,实验结果表明:文中所提方法具有很好的平移,旋转和缩放不变性,能适应复杂环境的变化.     

Novel Haar features for real-time hand gesture recognition using SVM

Journal of Real-Time Image Processing - Due to the effect of lighting and complex background, most visual hand gesture recognition systems work only under restricted environments. Here, we propose...     

Emotion-relevant activity recognition based on smart cushion using multi-sensor fusion

More and more common activities are leading to a sedentary lifestyle forcing us to sit several hours every day. In-seat actions contain significant hidden information, which not only reflects the current physical health status but also can report mental states. Considering this, we design a system, based on body-worn inertial sensors (attached to user’s wrists) combined with a pressure detection module (deployed on the seat), to recognise and monitor in-seat activities through sensor- and feature-level fusion techniques. Specifically, we focus on four common basic emotion-relevant activities (i.e. interest-, frustration-, sadness- and happiness-related). Our results show that the proposed method, by fusion of time- and frequency-domain feature sets from all the different deployed sensors, can achieve high accuracy in recognising the considered activities.     

Two-handed gesture recognition and fusion with speech to command a robot

Assistance is currently a pivotal research area in robotics, with huge societal potential. Since assistant robots directly interact with people, finding natural and easy-to-use user interfaces is of fundamental importance. This paper describes a flexible multimodal interface based on speech and gesture modalities in order to control our mobile robot named Jido. The vision system uses a stereo head mounted on a pan-tilt unit and a bank of collaborative particle filters devoted to the upper human body extremities to track and recognize pointing/symbolic mono but also bi-manual gestures. Such framework constitutes our first contribution, as it is shown, to give proper handling of natural artifacts (self-occlusion, camera out of view field, hand deformation) when performing 3D gestures using one or the other hand even both. A speech recognition and understanding system based on the Julius engine is also developed and embedded in order to process deictic and anaphoric utterances. The second contribution deals with a probabilistic and multi-hypothesis interpreter framework to fuse results from speech and gesture components. Such interpreter is shown to improve the classification rates of multimodal commands compared to using either modality alone. Finally, we report on successful live experiments in human-centered settings. Results are reported in the context of an interactive manipulation task, where users specify local motion commands to Jido and perform safe object exchanges.