A Tracking Registration Method for Augmented Reality Based on Multi-modal Template Matching and Point Clouds

In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality, a markerless augmented reality tracking registration method based on multimodal template matching and point clouds is proposed. The method first adapts the linear parallel multi-modal LineMod template matching method with scale invariance to identify the texture-less target and obtain the reference image as the key frame that is most similar to the current perspective. Then, we can obtain the initial pose of the camera and solve the problem of re-initialization because of tracking registration interruption. A point cloud-based method is used to calculate the precise pose of the camera in real time. In order to solve the problem that the traditional iterative closest point (ICP) algorithm cannot meet the real-time requirements of the system, Kd-tree (k-dimensional tree) is used under the graphics processing unit (GPU) to replace the part of finding the nearest points in the original ICP algorithm to improve the speed of tracking registration. At the same time, the random sample consensus (RANSAC) algorithm is used to remove the error point pairs to improve the accuracy of the algorithm. The results show that the proposed tracking registration method has good real-time performance and robustness.

     

An FPGA stereo matching unit based on fuzzy logic

Stereo matching is one of the most used algorithms in real-time image processing applications such as positioning systems for mobile robots, three-dimensional building mapping and recognition, detection and three-dimensional reconstruction of objects. In order to improve the performance, stereo matching algorithms often have been implemented in dedicated hardware such as FPGA or GPU devices. In this paper an FPGA stereo matching unit based on fuzzy logic is described. The proposed algorithm consists of three stages. First, three similarity parameters inherent to each pixel contained in the input stereo pair are computed. Then, the similarity parameters are sent to a fuzzy inference system which determines a fuzzy-similarity value. Finally, the disparity value is defined as the index which maximizes the fuzzy-similarity values (zero up to d_max). Dense disparity maps are computed at a rate of 76 frames per second for input stereo pairs of 1280 × 1024 pixel resolution and a maximum expected disparity equal to 15. The developed FPGA architecture provides reduction of the hardware resource demand compared to other FPGA-based stereo matching algorithms: near to 72.35% for logic units and near to 32.24% for bits of memory. In addition, the developed FPGA architecture increases the processing speed: near to 34.90% pixels per second and outperforms the accuracy of most of real-time stereo matching algorithms in the state of the art.     

基于YOLOv3的嵌入式实时视频目标检测算法

深度神经网络在目标检测领域具有优异的检测性能,但其结构复杂、计算量大,难以在嵌入式设备上进行高性能的实时目标检测。针对该问题,提出一种基于YOLOv3的目标检测算法。采用半精度推理策略提高YOLO算法的推理速度,并通过视频运动自适应推理策略充分利用前后帧视频之间目标的关联性,降低深度学习算法的运行频率,进一步提高目标检测速度。在ILSVRC数据集上的实验结果表明,该算法可以在NVIDIA TX2嵌入式平台上实现28 frame/s的视频目标检测,且检测精度与原始的YOLOv3算法相当。     

DSP处理器上的高效串匹配实现

字符串匹配是生物识别、入侵检测的基础,也是大数据互联网时代的研究热点.随着现代信息技术的发展,日常工作生活中移动及手持小型化设备的使用越发普遍.这些设备的应用场景中包含大量有关串匹配的需求,如人脸识别、实时数据查询等.串匹配算法的实时和准确性决定了使用场景的范围,因此在DSP处理器等移动小型化设备的嵌入式处理器上实现高效串匹配算法的问题变得十分迫切.该文针对DSP处理器因缺乏逻辑判断与跳转指令,难以支持高效串匹配运算的问题,提出了一种基于DSP平台特点的改进串匹配算法.该算法采用位并行的思路,在DSP处理器上实现了串匹配算法的并行化.同时通过前序启动、基于VLIW的数学运算替代逻辑判断、Q-grams等优化手段,提高该算法对于DSP平台的适应性与执行效率,最终实现了一种基于HXDSP的高效串匹配算法VBNDM2.实验结果表明,本算法针对DSP平台,有效地提高了串匹配的效率,实现了算法的高效并行化.     

采用快速SIFT算法实现目标识别①

在基于原始SIFT算法的目标识别中,特征描述符的计算复杂,特征点的匹配时间较长,为此提出一种快速SIFT算法。该算法采用同心圆形窗口内的灰度累加值和差分值构建16维的简化描述符,并且在目标识别时,按照金字塔结构由粗至精进行特征点匹配。实验表明,在保证目标识别准确率的前提下,快速算法的运算时间比原始SIFT算法减少了两个数量级,具有很好的实时性能。     

基于谱残差和聚类法的运动目标检测研究

传统基于特征点匹配的目标检测算法目标识别率低、误检率较高是因为特征点匹配不准确、目标轮廓不连续。针对这一问题,分别引入谱残差算法和k means聚类算法,并加以改进,提出一种基于谱残差算法和k means聚类算法的运动目标检测算法。具体方法是：首先,每隔两帧提取加速鲁棒特征SURF并对图像配准,再对帧差结果采用谱残差算法提取视觉显著性特征,去除因匹配不准确造成的噪点和伪运动目标;其次,形态学处理之后引入改进后的k means聚类算法,对不连续的轮廓进行聚类;最后形成完整的目标。实验显示,本文算法目标识别率达到90.61%,误检率达到21.25%,分别优于传统基于SURF特征的运动目标检测算法66.60%的识别率、31.91%的误检率和基于新的局部不变性特征ORB匹配的目标检测算法87.573%的识别率、26.80%的误检率。虽然该算法平均运行时间为18 fps,但仍可以满足视频流畅的需求,因此动态背景下该算法可做为一种有效的运动目标检测算法使用。     

基于轻量级OpenPose改进的幻影机手势交互系统

目前人机交互方式多以键盘鼠标为主,而基于深度学习手势识别的交互方式算法准确率不高,且实时性和系统稳定性均有待提升。提出一种新颖的针对轻量级OpenPose进行改进的幻影机手势交互系统。采用轻量级OpenPose将人手简化建模为21个关键点,以MobileNetV1作为基础模型,应用部分亲和域（Part Affinity Fields,PAF）方法实现人手关键点的检测并画出简化骨骼图。为进一步提升人机交互系统的实时性,采用幻影模块（Ghost Module）对卷积层进行降维,用更少的硬件资源取得同样的识别效果。最后搭建验证环境,根据画出的人手骨骼图进行模式匹配,根据匹配识别结果生成交互控制指令,经由蓝牙通讯将指令传送至Arduino UNO平台控制小车实现交互响应。经过初步训练后,该系统在COCO2017验证集上能实现58.7%的准确率,保持了原始OpenPose网络和轻量OpenPose网络的人手关键点识别效果,在家用PC机上可实现每秒32~36帧的识别速率和较高的手势识别率。     

A Real-Time Image Recognition System for Tiny Autonomous Mobile Robots

Intelligent sensors for mobile robots play an important role in many technical applications. In this paper a real-time image recognition system for a tiny autonomous mobile robot is presented, capable of detecting objects in real-time at a frame rate of up to 60 frames/s. The image recognition module has very low power consumption of less than 250 mW and fits into a package of only 35 × 35 mm including a CMOS camera and a low power, high performance signal processor. We propose an object recognition algorithm that is optimized for deeply embedded systems used in energy and performance constrained devices. The algorithm is based on a combination of edge and color detection and uses a fixed model for each object to be recognized. Results of the ball recognition application show that its relative polar coordinates are found within 11 ms.Stefan Mahlknecht received a Masters degree in electrical engineering and a doctoral degree from Vienna University of technology. His doctoral thesis dealt with the topic of energy selfsufficient wireless sensor networks. He spend one year at the University of Illinois in Urbana Champain, where he focused on communication networks and protocols. Since 2001 he is a member of the departements staff. His research interests are devoted to the topic of wireless sensor networks, communication protocols, embedded systems and robotics.Roland Oberhammer is a master thesis student at the Institute of Computer Technology at the Vienna University of Technology. His master thesis dealt with the topic of real-time ball recognition on resource limited embedded platforms. His research interests are in the field of embedded systems and robotics.Gregor Novak received a Masters degree in mechanical engineering and a doctoral degree in technical sciences both from the Vienna University of Technology in Austria in the years 1997 and 2002, respectively as well as a Masters degree in engineering management in 2001 from Oakland University in USA. Presently he is the coordinator of the Vienna University of Technologys Center of Excellence for Autonomous Systems. His research focuses on autonomous mobile cooperating robots.     

基于Ncut的SIFT特征匹配算法

经典尺度不变特征变换(SIFT)特征匹配算法存在实时性差、纹理相似区域易发生误匹配的问题。为此,提出一种基于归一化分割(Ncut)的SIFT特征匹配算法。针对相同背景的运动视频,将归一化分割算法的图论聚类思想融入SIFT特征匹配中,根据运动趋势相似度对特征点进行Ncut运动聚类,再逐类分别匹配,通过缩小各特征点匹配过程中的搜索范围,减少匹配时间及不同特征类之间的误匹配。实验结果表明,该算法能提高匹配效率,对纹理相似区域的误匹配现象有较好的抑制作用,实现了相邻图像帧的特征稳定匹配。     

一种基于几何特征的改进模板匹配算法

在线视觉棒材计数系统通过实时处理视频信息对运动对象进行计数，为了满足实时性行要求，本文提出一种基于几何特征的快速模板匹配算法，实验表明该算法能有效地识别出棒材目标，并显著提高了速度。     

面向移动设备的快速特征点匹配方法研究

针对增强现实在移动设备应用中特征点匹配速度缓慢的情况,提出了一种特征点匹配优化算法.该算法基于移动设备自带的重力感应系统,用重力方向代替原有的描述符方向,以此来降低传统ORB(oriented FASTand rotated BRIEF)算法中特征点方向角计算的时间复杂度.在实际移动设备上的测试结果表明,改进后的算法同时提高了匹配速度和匹配准确率,具有较好的效果.     

结合图像识别的移动增强现实系统设计与应用

目的 在移动互联网时代下,移动增强现实应用得到越来越快的发展。然而户外场景中存在许多相似结构的建筑,且手机的存储和计算能力有限,因此应用多集中于室内小范围环境,对于室外大规模复杂场景的适应性较弱。对此,建立一套基于云端图像识别的移动增强现实系统。方法 为解决相似特征的误匹配问题,算法中将重力信息加入到SURF和BRISK特征描述中去,构建Gravity-SURF和Gravity-BRISK特征描述。云端系统对增强信息进行有效管理,采用基于Gravity-SURF特征的VLAD方法对大规模图像进行识别;在智能终端上的应用中呈现识别图像的增强信息,并利用识别图像的Gravity-BRISK特征和光流结合的方法对相机进行跟踪,采用Unity3D渲染引擎实时绘制3维模型。结果 在包含重力信息的4 000幅户外图像的数据库中进行实验。采用结合重力信息的特征描述算法,能够增强具有相似特征的描述符的区分性,并提高匹配正确率。图像识别算法的识别率能达到88%以上,识别时间在420 ms左右;光流跟踪的RMS误差小于1.2像素,帧率能达到23 帧/s。结论 本文针对室外大规模复杂场景建立的基于图像识别的移动增强现实系统,能方便对不同应用的增强现实数据进行管理。系统被应用到谷歌眼镜和新闻领域上,不局限于单一的应用领域。结果表明,识别算法和跟踪注册算法能够满足系统的精度和实时性要求。     

一种基于几何特征的改进模板匹配算法

在线视觉棒材计数系统通过实时处理视频信息对运动对象进行计数,为了满足实时性行要求,本文提出一种基于几何特征的快速模板匹配算法,实验表明该算法能有效地识别出棒材目标,并显著提高了速度。     

基于TLD框架的快速目标跟踪方法

现实中目标在被长期跟踪时容易发生形变、遮挡、光照干扰以及其它问题,现有跟踪算法虽能解决该系列问题但算法计算量巨大导致跟踪系统实时性能较差,很难应用于实际场合。因此准确快速跟踪目标成为近年来非常有挑战的热点课题。以国外学者Zdenek Kalal等人提出的TLD(Tracking-Learning-Detection)框架为基础,提出了三点改进方法。一根据目标所占整幅图像的面积大小动态调整被处理图像的分辨率,从总体上减少样本数量;二在目标邻近区域扫描生成样本,缩小检测器的检测范围;三更换检测部分中分类器模板匹配方法,实现快速匹配,提高算法运行速度。针对与不同的场景,实验表明上述问题在改进后的算法中得到了较大的改善,算法的计算量有效降低,系统运行速度得到提高。且对于实时摄像头监控,改进后算法在保证目标跟踪准确率的同时拥有较好的实时性。     

基于稀疏卷积特征和相关滤波的实时视觉跟踪算法

为提高分层卷积相关滤波视觉跟踪算法的实时性能,提出一种稀疏卷积特征的实时目标跟踪算法。首先,在分析不同层卷积特征的基础上,采用等间隔采样的方式提取每个卷积层的稀疏卷积特征;然后,对每个卷积层特征的相关滤波响应值进行加权组合,得到目标预测的位置;最后,采用稀疏的模型更新策略进一步提高算法的运行速度。在OTB-2015新增的50组数据上对所提算法进行测试,实验结果表明,该算法的平均距离精度为82.2%,比原分层卷积特征跟踪算法提高了5.25个百分点,对目标姿态以及遮挡等变化具有较好的鲁棒性。该算法的平均跟踪速度为32.6帧/s,是原分层卷积特征跟踪算法的近3倍,能达到实时跟踪的效果。     

Personness estimation for real-time human detection on mobile devices

One aim of detection proposal methods is to reduce the computational overhead of object detection. However, most of the existing methods have significant computational overhead for real-time detection on mobile devices. A fast and accurate proposal method of human detection called personness estimation is proposed, which facilitates real-time human detection on mobile devices and can be effectively integrated into part-based detection, achieving high detection performance at a low computational cost. Our work is based on two observations: (i) normed gradients, which are designed for generic objectness estimation, effectively generate high-quality detection proposals for the person category; (ii) fusing the normed gradients with color attributes improves the performance of proposal generation for human detection. Thus, the candidate windows generated by the personness estimation will very likely contain human subjects. The human detection is then guided by the candidate windows, offering high detection performance even when the detection task terminates prior to completion. This interruptible detection scheme, called anytime detection, enables real-time human detection on mobile devices. Furthermore, we introduce a new evaluation methodology called time-recall curves to practically evaluate our approach. The applicability of our proposed method is demonstrated in extensive experiments on a publicly available dataset and a real mobile device, facilitating acquisition and enhancement of portrait photographs (e.g. selfie) on widespread mobile platforms.     

基于区域匹配的实时加速技术

针对区域立体匹配计算量大实时性差的困难,分析了相关匹配算法的实际工作过程,采用消除冗余因子和Box滤波、多级分辨率匹配减小计算复杂度,对算法结构进行了改进和优化,并利用超线程和OpenMP技术对算法进行了加速,提出了一种实时区域匹配算法.对算法进行实验,结果表明算法符合了视觉导航的准确性和实时性要求,并且对于提高其他区域匹配算法实时性也具有重要借鉴意义.     

融合压缩感知和SVM的SAR变形目标识别算法

为降低合成孔 径雷达(Synthetic aperture radar, SAR)图像目标识别中目标方位角的影响,并提高对SAR变形目标的识别率,本文提出了一种基于压缩感知和支持向量机决策级融合的目标识别算法。该算法首先基于稀疏表征理论将SAR目标识别问题描述为压缩感知的稀疏信号恢复问题,然后基于稀疏系数分别进行目标类别判别与方位角估计。对样本进行姿态校正后,利用支持向量机分别对经过姿态校正和未经姿态校正的样本进行目标分类。最后采用投票表决法对3种算法的分类结果进行决策级融合。实验结果表明,基于压缩感知结果进行目标方位角估计有效,且随着训练样本数的增加,提出的决策级融合算法提高了SAR变形目标的识别率。     

基于NCS2神经计算棒的车辆检测方法

基于深度学习的车辆检测方法准确率较高,其在性能卓越的计算机与图形处理器设备上实时性较好,但在性能相对较低的嵌入式设备上实时性较差.在改进Tiny-YOLO网络的基础上,提出一种利用NCS2神经计算棒的嵌入式车辆检测方法.采用深度可分离卷积替换Tiny-YOLO网络标准卷积降低计算量,去除池化层并使用全卷积层以保留低级特...     

基于并行跟踪检测框架与深度学习的目标跟踪算法

在空地协同背景下，地面目标的移动导致其在无人机视角下外观会发生较大变化，传统算法很难满足此类场景的应用要求。针对这一问题，提出基于并行跟踪和检测（PTAD）框架与深度学习的目标检测与跟踪算法。首先，将基于卷积神经网络（CNN）的目标检测算法SSD作为PTAD的检测子处理关键帧获取目标信息并提供给跟踪子；其次，检测子与跟踪子并行处理图像帧并计算检测与跟踪结果框的重叠度及跟踪结果的置信度；最后，根据跟踪子与检测子的跟踪或检测状态来判断是否对跟踪子或检测子进行更新，并对图像帧中的目标进行实时跟踪。在无人机视角下的视频序列上开展实验研究和对比分析，结果表明所提算法的性能高于PTAD框架下最优算法，而且实时性提高了13%，验证了此算法的有效性。