Thermal image-based hand gesture recognition for worker-robot collaboration in the construction industry: A feasible study

Worker-robot collaboration (WRC) is a promising solution for complex construction tasks, which can integrate the robots’ advantages in strength and accuracy with human ability in intuitive decision-making and adaptability. A new imperative objective for real-world WRC is to design a user-friendly interface to support safe and efficient worker-robot interactions. Vision-based hand gesture is a simple but effective solution. However, existing methods mainly depend on 3-channel RGB images captured by visible cameras, which are prone to be affected by on-site environmental disturbances, such as poor illumination, fog, and dust. Moreover, previous networks strive for high accuracy, neglecting computational efficiency (e.g., model size and latency) when implementing the network on resource-constrained devices like mobile construction robots. Against this backdrop, this research presented a feasibility study to investigate whether hand signals can be detected from thermal images and designed a lightweight network that has fewer parameters and obtains lower latency without compromising accuracy. Experimental results indicated that thermal images were robust to different lighting conditions, and the proposed model achieved a high classification accuracy (97.54 %) with 1.8 M parameters. The comparative study demonstrated the superiority of our model to other advanced lightweight models, illustrating the feasibility of the developed method in supporting safe WRC applications by recognizing workers’ hand signals.     

动与静目标双重检测的手势检测方案及硬件设计

基于超声波的手势识别系统中引入手势检测方案一方面可以在无手势的时候不执行高复杂度的识别运算，有效地降低功耗；另一方面还可以分割出有效时间片段，提高手势识别的精度。为提升手势识别系统的性能，提出了一种手势检测方案并且以硬件实现了相应功能。采用动目标与静目标双重检测以及杂波图技术，提供手势识别系统在低功耗的检测模式以及高精度运算的识别模式间切换的条件，实现对环境静态杂波的跟踪和滤除，同时解决了传统单纯动目标检测在连续手势操作时存在的漏检问题。本方案由检测模式向识别模式切换的漏警率为0,识别模式向检测模式切换的虚警率为2%,模式切换的响应时间为733.47 ns。     

基于多特征加权融合的静态手势识别

针对人工提取特征的单一性及卷积神经网络提取特征的遗漏性问题,提出了一种基于多特征加权融合的静态手势识别方法.首先,提取分割后的手势图像的傅里叶和Hu矩等形状特征,将两者融合作为手势图像的局部特征;设计双通道卷积神经网络提取手势图像的深层次特征,采用主成分分析方法对提取的特征进行降维;然后,将提取的局部特征和深层次特征进...     

基于空中手势的跨屏幕内容分享技术研究

在多屏幕环境中，实现跨屏幕内容分享的现有技术忽视了对空中手势的利用.为验证空中手势，实现并部署了一套跨屏幕界面共享系统，支持使用空中手势完成屏幕间界面内容的实时分享.根据一项让用户设计手势的调研，实现了"抓-拖拽"和"抓-拉-放"两种空中手势.通过对两种手势进行用户实验发现：用空中手势完成跨屏幕内容分享是新颖、有用、易于掌握的.总体上，"抓-拖拽"手势用户感受更好，但"抓-拉-放"手势对于复杂情景的通用性更好；前者更适合近屏场景，推荐在屏幕相邻且间距不大、操作距离近时使用；后者更适合远屏场景，推荐在目标屏幕间有间隔屏幕或相距较远时使用.因此，将两种手势相结合使用更为合理.