基于Python的星际飞行大战游戏设计与实现

研究基于手势识别技术的鼠标操控方法,并将其应用“星际飞行大战”游戏中。通过使用深度学习模型进行手势识别,将用户的手势转化为鼠标的移动和点击操作。利用Python编程语言和开源库进行实现,并通过测试验证了该方法的准确性和可行性。结果表明,基于手势识别技术的鼠标操控方法可以有效地提高玩家的游戏体验和操作精度。未来,该方法还可以被应用于更广泛的应用场景。     

全面应用鼠标手势StrikeltPersonal

鼠标手势（MouseGesture）是一项通过识别手写输入来操控计算机的技术,较常见用于网页浏览器上。不过本期介绍的这款“Strokelt”软件却可以在整个系统的软件中全面支持鼠标手势,使用户在执行打开新文件、储存文件、复制及粘贴文件等操作时,都可以抛弃键盘,也不用再研究这些指令都“藏”在那个菜单内。此外“Strokelt”还具备学习功能,用户可以自己创建鼠标手势,把新建立的手势应用在不同类型的操作上。     

基于手势识别的虚实交互技术研究

随着虚拟现实技术的发展,人们对虚实交互需求日益增强,手势交互以其简单灵活的特点,成为非接触式人机交互的重要方式之一。为了解决传统手势交互方法精度低,鲁棒I性差的问题,提出了基于动态规划算法,I通过分析骨架信息,计算人体的运动轨迹,然后利用骨架的位置和距离信息,对手部图像进行RO选取,然后使用FasterR-CNN算法对RO特征提取,从而实现手势动作的细粒度识别。实验表明,手势识别的置信度和准确率由较大的提升,可准确识别小目标手势。     

一种基于Leap Motion的非参数RDP检测算法

随着计算机软硬件技术的进步和应用的普及,人机交互技术在博物馆领域中扮演着越来越重要的角色,并且受到了学术界和产业界的高度重视。尤其是Leap Motion体感控制器的出现,使人机交互的应用范围更加广泛与成熟,操作者可以通过非接触式的方式对设备进行操作,而无需使用触控屏、鼠标、键盘等外部设备,令人机交互方式更加友好、便捷。为了提高手势识别的准确性与实用性,提出一种基于Leap Motion的非参数RDP检测算法应用在手势识别中,并与Ramer-Douglas-Peucker(RDP)算法进行比较。实验证明使用非参数RDP检测算法可以有效地识别手势并且具有很好的自适应性。     

基于DTW算法的肌电信号手势识别方法

为了提高肌电信号手势识别算法的准确度,增强实时性,提出了一种基于动态时间规整（DTW）算法的手势识别方法,该方法利用肌电信号（EMG）对个体间的手势进行识别。首先,采用滑动平均能量的方法对原始的EMG信号进行数据分割,探测有效动作;其次,对于分割的数据段使用平均绝对值（MAV）来提取信号特征;最后,用DTW算法将8维的EMG信号融合并计算测试样本和模版的相似度,其中采用了DTW算法寻找规整路径的方法进行了模板制作,实现了个体间的手势识别。实验结果表明,使用DTW算法对肌电信号进行手势识别,其动作识别的准确率达到96.09%,该方法计算速度快,实时性强。     

基于多传感器融合的动态手势识别研究分析

研究利用三类传感器（表面肌电仪、陀螺仪和加速度计）信号的特点进行信息融合,提高可识别动态手势动作的种类和准确率。将动态手势动作分解为手形、手势朝向和运动轨迹三个要素,分别使用表面肌电信号（sEMG）、陀螺仪信号（GYRO）和加速度信号（ACC）进行表征,利用多流HMMs进行动态手势动作的模式识别。对包含有5个运动轨迹和6个静态手形的识别实验结果表明,该方法可以有效地从连续信号中识别动态手势,三类传感器组合使用获得的全局平均识别率达到92％以上,明显高于任意两个传感器组合和仅采用单个传感器获得的平均识别率。实验表明该方法是一种有效的动态手势识别方法,并且相较于传统的动态手势识别的方法更具有优势。     

基于手势识别远程同步控制的智能机械臂研究

设计一种基于手势识别远程同步控制的智能机械臂系统,系统由手势识别器和智能机械臂组成;手势识别器穿戴在手中,能够感知和识别手势指令,并通过无线传输方式把手势指令传输给智能机械臂;智能机械臂收到手势指令后其自主决策系统迅速规划最优控制,实现同步控制机械臂伸展和机械手的抓取等动作;实验结果表明,手势识别器的手势指令简单易于操作,手势指令识别准确高效,智能机械臂动作规划协调,系统大大降低了机械臂的操控难度,完全满足作业任务实时控制的需要,具有较好的实用性和广泛的应用前景.     

基于OpenCV的机器视觉在智能手机中的应用

针对智能手机市场竞争中主要力拼软件的特点,将机器视觉技术引入以Linux为操作系统的智能手机中,基于Open-CV研究并实现了手势识别控制应用程序。系统决策实现的部分是以手势来启动音乐播放程序进行讨论,而且程序留有扩展接口可以实现更多不同的手势来控制不同的操作。实验结果表明,该系统满足实时处理需求,运行稳定。这种非接触式控制智能手机操作的功能既实用又前卫,使Linux操作系统的智能手机更具吸引力,发展前景更广阔。     

基于手势特征融合的操作动作识别

针对动态复杂场景下的操作动作识别,提出一种基于手势特征融合的动作识别框架,该框架主要包含RGB视频特征提取模块、手势特征提取模块与动作分类模块。其中RGB视频特征提取模块主要使用I3D网络提取RGB视频的时间和空间特征;手势特征提取模块利用Mask R-CNN网络提取操作者手势特征;动作分类模块融合上述特征,并输入到分类器中进行分类。在EPIC-Kitchens数据集上,提出的方法识别抓取手势的准确性高达89.63%,识别综合动作的准确度达到了74.67%。     

基于Leap Motion的动态手势识别方法

伴随虚拟现实（Virtual Reality,VR）技术的发展,以及人们对人机交互性能和体验感的要求提高,手势识别作为影响虚拟现实中交互操作的重要技术之一,其精确度急需提升^[1].针对当前手势识别方法在一些动作类似的手势识别中表现欠佳的问题,提出了一种多特征动态手势识别方法.该方法首先使用体感控制器Leap Motion追踪动态手势获取数据,然后在特征提取过程中增加对位移向量角度和拐点判定计数的提取,接着进行动态手势隐马尔科夫模型（Hidden Markov Model,HMM）的训练,最后根据待测手势与模型的匹配率进行识别.从实验结果中得出,该多特征识别方法能够提升相似手势的识别率.     

The design and empirical evaluations of 3D positioning techniques for pressure-based touch control on mobile devices

The previous three degrees of freedom (DOF) 3D touch translations require more than one finger (usually two hands) to be performed, which limits their usability on mobile devices that need one hand to be held in most occasions. Given that the pressure-sensitive touch screen will become ubiquitous in the near future, we presented a pressure-based 3DOF 3D positioning technique that only uses one finger in operating. Our technique collects the normal force of the touch pressure and uses it to represent the depth value in 3D translating. Then we conducted several groups of tightly controlled user studies to conclude (1) how different strategies of pressure recognition will affect 3D translating and (2) how is the performance of the pressure-based manipulation compared to the previous two-fingered technique. Finally, we discussed some guidelines to help developers in the design of the pressure-sensing technique in 3D manipulations.     

Three‐dimensional interaction and autostereoscopic display system using gesture recognition

We propose a 3D interaction and autostereoscopic display system that use gesture recognition, which can manipulate virtual objects in the scene directly by hand gestures and can display objects in 3D stereoscopy. The system consists of a gesture recognition and manipulation part as well as an autostereoscopic display as an interactive display part. To manipulate the 3D virtual scene, a gesture recognition algorithm is proposed, which use spatial‐temporal sequences of feature vectors to match predefined gestures. To get smooth 3D visualization, we utilize the programmable graphics pipeline in graphic processing unit to accelerate data processing. We develop a prototype system for 3D virtual exhibition. The prototype system reaches frame rates of 60 fps and operates efficiently with a mean recognition accuracy of 90%.     

非接触式触摸屏系统的设计

针对目前触摸屏必须接触式控制的缺陷,提出了一种基于图像处理技术的新型触摸屏系统。该系统以微软VX—6000USB2.0摄像头为图像传感器,由PC机采集图像序列,识别和跟踪定位激光光斑,并对用户的指点控制信息做出相应反应。阐述了该系统结构及定位原理,使用基于OpenCV（open source computer vision library）的图像处理算法辅助开发其软件系统。提出一种基于边界修复的图像二次校正法,通过对训练样本的测量发现,系统拥有很好的指点精度。实验结果表明,所设计触摸屏系统能够初步实现非接触式指点的要求,为大屏幕挂壁式非接触式触摸屏系统的研究提供了有效的思路。     

Computing with feeling

Computers are usually considered as manipulators of numbers, words and symbols: machines with which we communicate visually by printers, plotters and displays, and sometimes audibly through music synthesis and speech recognition. We are engaged in a project involving touch communication with computers in order to broaden computer graphics capability. Originally pioneered by Noll and by Batter and Brooks, touch communication with computers can link together brain, hands and computer to explore new worlds of felt imagery, worlds existing only in computer memory.Humans are distinguished from other animals particularly by two things: the hands, which allow fine manipulations of objects in the environment, and the brain, which permits thought. Touch communication with computers employs both of these most human capabilities. What we're searching for is a closer symbiosis between humans and machines, a partnership of two unlike species growing together as both learn to perform joint tasks better. This symbiosis demands a better interface where machine meets person.The first of our touch communication systems is a three-dimensional force-position system called “Touchy Feely” in which mechanical simplicity is gained by using a tetrahedral coordinate system, employing the computer to transform into other coordinate systems. We are also designing a force, torque, position and orientation system. “Touchy Twisty”, which will permit the user to feel the docking of one three-dimensional object with another: in other words, to allow the assembly of computer simulated objects.There are many applications of human-computer touch communication to research and learning, extending into such areas as computer science, engineering design, chemistry, physics, biology, medicine, psychology, art, and insight for the blind. With touch communication we can feel things never felt or seen before and perceive spatial relationships not otherwise possible. We can thereby create a more sensitive awareness and understanding of natural phenomena in three-dimensional space, phenomena involving forces and torques for which visual representation is often inappropriate or impossible.     

Hand gesture recognition from depth and infrared Kinect data for CAVE applications interaction

This paper presents a real-time framework that combines depth data and infrared laser speckle pattern (ILSP) images, captured from a Kinect device, for static hand gesture recognition to interact with CAVE applications. At the startup of the system, background removal and hand position detection are performed using only the depth map. After that, tracking is started using the hand positions of the previous frames in order to seek for the hand centroid of the current one. The obtained point is used as a seed for a region growing algorithm to perform hand segmentation in the depth map. The result is a mask that will be used for hand segmentation in the ILSP frame sequence. Next, we apply motion restrictions for gesture spotting in order to mark each image as a ‘Gesture’ or ‘Non-Gesture’. The ILSP counterparts of the frames labeled as “Gesture” are enhanced by using mask subtraction, contrast stretching, median filter, and histogram equalization. The result is used as the input for the feature extraction using a scale invariant feature transform algorithm (SIFT), bag-of-visual-words construction and classification through a multi-class support vector machine (SVM) classifier. Finally, we build a grammar based on the hand gesture classes to convert the classification results in control commands for the CAVE application. The performed tests and comparisons show that the implemented plugin is an efficient solution. We achieve state-of-the-art recognition accuracy as well as efficient object manipulation in a virtual scene visualized in the CAVE.     

基于轻量级OpenPose改进的幻影机手势交互系统

目前人机交互方式多以键盘鼠标为主,而基于深度学习手势识别的交互方式算法准确率不高,且实时性和系统稳定性均有待提升。提出一种新颖的针对轻量级OpenPose进行改进的幻影机手势交互系统。采用轻量级OpenPose将人手简化建模为21个关键点,以MobileNetV1作为基础模型,应用部分亲和域（Part Affinity Fields,PAF）方法实现人手关键点的检测并画出简化骨骼图。为进一步提升人机交互系统的实时性,采用幻影模块（Ghost Module）对卷积层进行降维,用更少的硬件资源取得同样的识别效果。最后搭建验证环境,根据画出的人手骨骼图进行模式匹配,根据匹配识别结果生成交互控制指令,经由蓝牙通讯将指令传送至Arduino UNO平台控制小车实现交互响应。经过初步训练后,该系统在COCO2017验证集上能实现58.7%的准确率,保持了原始OpenPose网络和轻量OpenPose网络的人手关键点识别效果,在家用PC机上可实现每秒32~36帧的识别速率和较高的手势识别率。     

基于支持向量机的抽油机井功图识别研究

抽油机的异常情况会使油田的产油效率降低,而不同的异常类型对应的抽油机示功图特征也各不相同,因此造成的损害程度也不同。针对以上问题,文中提出了一种抽油机井功图识别模型,该方法将支持向量机（ SVM）用于抽油机井功图识别。首先利用改进的矢量曲线数据压缩方法（ICVDC）对抽油机井下示功图进行特征数据提取,在此基础上,采用“一对一”分类法建立基于支持向量机的井下示功图分类模型,进而对不同特征的示功图进行分类识别,并与其他识别分类模型进行了识别分类效果对比。实验结果表明,该方法分类准确度高,有效地解决了示功图的识别和分类问题,方便对油井设备等进行进一步的故障分析处理,从而大大提高抽油机的性能与效率,以此来达到油田提高采收率的目的。     

Effects of Switchable DOF for Mid-Air Manipulation in Immersive Virtual Environments

In the context of mid-air manipulation, this paper presents the effects of allowing the user to dynamically switch between 1DOF, 2DOF, and 3DOF operations. Such “manipulation with switchable DOF” has been widely used in commercial graphics packages and is increasingly being adopted for virtual reality editing, where the 3D scene is constructed through mid-air interaction in immersive virtual environments. However, its effectiveness and advantages/disadvantages have not been investigated. This paper compares “manipulation with switchable DOF” with “manipulations with DOF separation,” which allows only 1DOF operations, and “manipulation without DOF separation,” which provides 3DOF operations. Using translation, rotation, and scaling, three methods were evaluated in terms of completion time and precision. The experiment results showed that “manipulation with switchable DOF” outperformed “manipulation with DOF separation” in terms of completion time whereas three methods were comparable in terms of precision. “Manipulation with switchable DOF” was further analyzed, and the results showed that the more 3DOF operations led to the shorter completion time and the more 1DOF operations led to the higher precision.