A Hierarchical Tracking Strategy for Vision-Based Applications On-Board UAVs

In this paper, we apply a hierarchical tracking strategy of planar objects (or that can be assumed to be planar) that is based on direct methods for vision-based applications on-board UAVs. The use of this tracking strategy allows to achieve the tasks at real-time frame rates and to overcome problems posed by the challenging conditions of the tasks: e.g. constant vibrations, fast 3D changes, or limited capacity on-board. The vast majority of approaches make use of feature-based methods to track objects. Nonetheless, in this paper we show that although some of these feature-based solutions are faster, direct methods can be more robust under fast 3D motions (fast changes in position), some changes in appearance, constant vibrations (without requiring any specific hardware or software for video stabilization), and situations in which part of the object to track is outside of the field of view of the camera. The performance of the proposed tracking strategy on-board UAVs is evaluated with images from real-flight tests using manually-generated ground truth information, accurate position estimation using a Vicon system, and also with simulated data from a simulation environment. Results show that the hierarchical tracking strategy performs better than well-known feature-based algorithms and well-known configurations of direct methods, and that its performance is robust enough for vision-in-the-loop tasks, e.g. for vision-based landing tasks.     

Omnivision-based KLD-Monte Carlo Localization

Mobile robots operating in real and populated environments usually execute tasks that require accurate knowledge on their position. Monte Carlo Localization (MCL) algorithms have been successfully applied for laser range finders. However, vision-based approaches present several problems with occlusions, real-time operation, and environment modifications. In this article, an omnivision-based MCL algorithm that solves these drawbacks is presented. The algorithm works with a variable number of particles through the use of the Kullback–Leibler divergence (KLD). The measurement model is based on an omnidirectional camera with a fish-eye lens. This model uses a feature-based map of the environment and the feature extraction process makes it robust to occlusions and changes in the environment. Moreover, the algorithm is scalable and works in real-time. Results on tracking, global localization and kidnapped robot problem show the excellent performance of the localization system in a real environment. In addition, experiments under severe and continuous occlusions reflect the ability of the algorithm to localize the robot in crowded environments.     

Fast Electronic Digital Image Stabilization for Off-Road Navigation

Image stabilization can be used as front-end system for many tasks that require dynamic image analysis, such as navigation and tracking of independently moving objects from a moving platform. We present a fast and robust electronic digital image stabilization system that can handle large image displacements based on a two-dimensional feature-based multi-resolution motion estimation technique. The method tracks a small set of features and estimates the movement of the camera between consecutive frames. Stabilization is achieved by combining all motion from a reference frame and warping the current frame back to the reference. The system has been implemented on parallel pipeline image processing hardware (a Datacube MaxVideo 200) connected to a SUN SPARCstation 20/612 via a VME bus adaptor. Experimental results using video sequences taken from a camera mounted on a vehicle moving on rough terrain show the robustness of the system while running at approximately 20 frames/s.     

夜间环境交通数据采集系统设计与实现

针对夜间交通环境的特点,设计和实现了一种基于车灯的交通流视频检测系统。首先,提出一种夜间车道线检测算法,提取车道线并标定摄像机参数。接着,采用一种自适应阈值分割算法提取候选车灯连通域,并利用空间距离信息配对和分组属于同一辆车的连通域,根据规则集定位车灯,建立车辆假设。然后,通过线性搜索,结合最近邻准则和形状属性匹配在帧间关联车辆假设。对于部分和全部遮挡的情况,结合Kalman滤波器处理。根据跟踪信息的连续性,确认车辆存在并保存跟踪轨迹。实验表明,算法的复杂度低,能够在夜晚多种交通环境下实时检测和跟踪车辆,误检和漏检率低,并且对遮挡情况具有一定的鲁棒性。     

ASSET-2: real-time motion segmentation and shape tracking

This paper describes a system for detecting and tracking moving objects in a moving world. The feature-based optic flow field is segmented into clusters with affine internal motion which are tracked over time. The system runs in real-time, and is accurate and reliable     

基于两步跟踪算法的海关车辆监控研究*

针对实时性和鲁棒性要求比较高的海关卡口车辆视频监控问题,提出了一种基于改进的均值漂移算法和粒子滤波算法的两步跟踪算法。对海关车辆监控的目标图像采用YCbCr颜色空间建立初始帧目标模型,利用改进后的均值漂移算法找出候选目标,在跟踪相似度小于设定的阈值时再利用改进后的粒子滤波算法进行后续的跟踪。通过实验分析,验证了提出的算法既能保证均值漂移算法跟踪的实时性,也能保证粒子滤波算法跟踪的鲁棒性,具有较好的应用价值。     

混合动力电动汽车的跟车控制与能量管理

混合动力电动汽车(Hybrid electric vehicles,HEVs)的能量管理问题至关重要,而混合动力电动汽车的跟车控制不仅涉及跟车效果与安全性,也影响着能量的高效利用.将HEVs的跟车控制与能量管理相结合,提出一种基于安全距离的HEVs车辆跟踪与能量管理控制方法.首先,考虑坡度、载荷变动建立了HEVs车辆跟...     

A Fuzzy-Logic-Based Approach for Mobile Robot Path Tracking

One important problem in autonomous robot navigation is the effective following of an unknown path traced in the environment in compliance with the kinematic limits of the vehicle, i.e., bounded linear and angular velocities and accelerations. In this case, the motion planning must be implemented in real-time and must be robust with respect to the geometric characteristics of the unknown path, namely curvature and sharpness. To achieve good tracking capability, this paper proposes a path following approach based on a fuzzy-logic set of rules which emulates the human driving behavior. The input to the fuzzy system is represented by approximate information concerning the next bend ahead the vehicle; the corresponding output is the cruise velocity that the vehicle needs to attain in order to safely drive on the path. To validate the proposed algorithm two completely different experiments have been run: in the first experiment, the vehicle has to perform a lane-following task acquiring lane information in real-time using an onboard camera; in the second, the motion of the vehicle is obtained assigning in real-time a given time law. The obtained results show the effectiveness of the proposed method     

基于概率算法自适应更新背景的运动车辆检测

交通流量检测是智能交通系统中的一个重要研究方向和热点问题,基于视频的车辆检测是交通流量采集分析的核心技术,它为交通流量参数的实时获取提供了可能。为实现在复杂交通视频场景中实时准确检测各类的运动车辆,在研究传统背景差分算法的缺点的工作基础上,提出一个自适应的贝叶斯概率背景检测算法,进而完成了较准确的运动车辆分类检测。实验结果表明该方法具有高效实时的特点,能够较准确地实现复杂交通路面的背景提取和运动车辆的检测,具有良好的鲁棒性。     

Robust object matching for persistent tracking with heterogeneous features

This paper addresses the problem of matching vehicles across multiple sightings under variations in illumination and camera poses. Since multiple observations of a vehicle are separated in large temporal and/or spatial gaps, thus prohibiting the use of standard frame-to-frame data association, we employ features extracted over a sequence during one time interval as a vehicle fingerprint that is used to compute the likelihood that two or more sequence observations are from the same or different vehicles. Furthermore, since our domain is aerial video tracking, in order to deal with poor image quality and large resolution and quality variations, our approach employs robust alignment and match measures for different stages of vehicle matching. Most notably, we employ a heterogeneous collection of features such as lines, points, and regions in an integrated matching framework. Heterogeneous features are shown to be important. Line and point features provide accurate localization and are employed for robust alignment across disparate views. The challenges of change in pose, aspect, and appearances across two disparate observations are handled by combining a novel feature-based quasi-rigid alignment with flexible matching between two or more sequences. However, since lines and points are relatively sparse, they are not adequate to delineate the object and provide a comprehensive matching set that covers the complete object. Region features provide a high degree of coverage and are employed for continuous frames to provide a delineation of the vehicle region for subsequent generation of a match measure. Our approach reliably delineates objects by representing regions as robust blob features and matching multiple regions to multiple regions using Earth Mover's Distance (EMD). Extensive experimentation under a variety of real-world scenarios and over hundreds of thousands of Confirmatory Identification (CID) trails has demonstrated about 95 percent accuracy in vehicle reacquisition with both visible and Infrared (IR) imaging cameras.     

基于车尾中轴特征的粒子滤波跟踪算法

为了解决车载视频背景实时变化的情况下车辆检测和跟踪问题,提出了一种基于车尾中轴特征的车辆识别及跟踪算法,其特点在于采用新颖的车尾中轴特征。车尾中轴特征具体为以两盏刹车灯为端点,车牌在端点连线的对称轴上。算法在取得目标区域之后计算其直方图信息,进行粒子滤波器跟踪。最后测试了算法的可行性。     

基于MSPF的实时监控多目标跟踪算法研究

近年来,实时监控下多目标跟踪作为智能交通系统(Intelligent transportation system,ITS)的重要组成部分受到关注.传统多目标跟踪方法通常具有处理速度慢、容易对交叉行进车辆产生误匹配等问题.本文首先对基于贝叶斯规则的车辆视频复杂背景的建模及运动目标的检测进行研究,在此基础上提出一种基于Meanshift粒子滤波(Mean shift particle filter,MSPF)的多目标跟踪算法,首先对每一目标车辆在下一帧可能出现的范围进行预测,对单目标和多目标情况采用不同的检测策略,避免了全局搜索,提高了跟踪速度;通过构造基于最新观测信息的重要性密度函数,提高了MSPF算法在复杂背景情况下追踪部分遮挡及交叉车辆的准确性和鲁棒性.仿真实验结果验证了所提出算法的有效性.     

宽基线主动视觉中感兴趣目标的对应技术

在主动视觉系统中,通常需要多个代理对同一场景中的感兴趣目标进行协同处理,以提高系统智能分析感兴趣目标的能力。其中,基于多视几何关系解决感兴趣目标的对应问题是协同处理的基础。一方面,主动视觉系统一般工作在宽基线条件下,这增加了对应问题描述的复杂性;另一方面,主动视觉系统以最佳视角观察目标,因此摄像头需做实时的姿态调整,由此导致的视间几何关系变化进一步加深了对应问题的解决难度。本文基于仿射不变的几何特征,建立宽基线条件下的多视几何关系,并针对频繁使用几何特征不能满足主动视觉系统实时要求的问题,提出一种快速更新多视几何关系的方法,并在多视几何约束下实现对应感兴趣目标的鲁棒标识。实验结果表明,该方法能解决宽基线主动视觉系统中感兴趣目标的复杂对应问题,并能达到实时要求。     

Optimal robust vehicle motion control under equality and inequality constraints

In this paper, a new robust controller is proposed to improve the motion control performance of an autonomous four-wheel steering vehicle. The vehicle is subject to time-varying uncertainties caused by parameter perturbation and unmodeled disturbance. First of all, the vehicle motion control problem is modeled as a constraint-following problem with the goal to drive the vehicle system to follow the given constraints. For safety reasons, inequality constraints are imposed on the lateral displacement of the vehicle. Next, the diffeomorphism mapping method is used to deal with the inequality constraints in the vehicle motion control, and the constrained lateral displacement space is mapped to an unbounded space. On this basis, a new multi-parameter robust constraint-following control is designed. Based on Lyapunov stability theory, the approximate constraint tracking performance of the path tracking system is proved. In order to make a trade-off between the system performance and control cost, a multi-parameter optimization problem is established, and the optimal robust controller is obtained. Last, the main theoretical results are verified by the Carsim-Simulink co-simulations.     

有界扰动下异质车辆队列分布式鲁棒经济预测控制

针对有界扰动下异质车辆队列节能与稳定分布式协同控制问题,提出一种新的分布式鲁棒经济模型预测控制(economic model predictive control, EMPC)策略.首先采用不确定误差模型描述有界扰动下异质车辆队列纵向行驶动态特性,再应用tube思想对系统约束进行紧缩设计,补偿有界扰动对系统造成的不确定性.其次,采用局部车辆行驶能耗模型描述车辆队列分布式经济性能优化的有限时域最优控制问题,并利用传统跟踪性能指标设计附加稳定收缩约束函数.进一步,基于系统收缩原理,建立车辆队列闭环系统关于有界扰动的输入-状态稳定性条件.最后,通过与车辆队列传统分布式鲁棒模型预测控制策略的数值仿真对比结果验证了所提出策略的有效性和优越性.     

Nonlinear robust adaptive sliding mode control design for miniature unmanned multirotor aerial vehicle

This paper addresses the stability and tracking control problem of miniature unmanned multirotor aerial vehicle (MUMAV) in the presence of bounded uncertainty. The uncertainty may appear from unmodeled dynamics, underactuated property, input disturbance and flying environment. Nonlinear robust adaptive sliding mode control algorithm is designed by using Lyapunov function. Robust adaptation laws are designed to learn and compensate the bounded parametric uncertainty. Lyapunov analysis shows that the proposed algorithms can guarantee asymptotic stability and tracking control property of the linear and angular dynamics of MUMAV system. Compared with other existing control methods, the proposed design is very simple and easy to implement as it does not require multiple design steps, augmented auxiliary signals and exact bound of the uncertainty. Experimental results on a miniature unmanned quadrotor aerial vehicle are presented to illustrate of effectiveness of the proposed design for real-time applications.     

Integrated real-time vision system for vehicle control in non-structured environments

An integrated vision-based real-time vehicle control system is proposed, as implemented in the experimental vehicle ROMEO-3R. One of the applications developed, an active object-tracking method using generalized predictive control, is also described. The real-time computer-vision module is based on a multiprocessor, DSP-based system; fast processing is achieved by parallel execution of the main image-processing algorithms. A pan-and-tilt platform allows the tracking process to be decoupled from vehicle motion.     

Output Feedback Dynamic Surface Controller Design for Airbreathing Hypersonic Flight Vehicle

This paper addresses issues related to nonlinear robust output feedback controller design for a nonlinear model of airbreathing hypersonic vehicle. The control objective is to realize robust tracking of velocity and altitude in the presence of immeasurable states, uncertainties and varying flight conditions. A novel reduced order fuzzy observer is proposed to estimate the immeasurable states. Based on the information of observer and the measured states, a new robust output feedback controller combining dynamic surface theory and fuzzy logic system is proposed for airbreathing hypersonic vehicle. The closedloop system is proved to be semi-globally uniformly ultimately bounded (SUUB), and the tracking error can be made small enough by choosing proper gains of the controller, filter and observer. Simulation results from the full nonlinear vehicle model illustrate the effectiveness and good performance of the proposed control scheme.      

智能车跟随系统开发

开发了一种基于STM32视觉导航的目标跟随小车智能控制系统,实现了对特定目标物体的快速识别与实时准确跟随.使用OV7670图像传感器采集目标物图像信息,对图像进行灰度化、降噪、取反等处理后,通过亚像元定位技术的质心法获取目标物坐标位置,为智能车提供方向导航,跟随过程中的画面实时发送至上位机.结果表明:小车在直行和转向跟踪实验中表现良好,跟随系统具有良好的跟踪效果,可实现对目标物的实时准确跟随.     

Real-time face tracking and recognition using the mobile robots

The terrain is complex and dynamic in a disaster. This paper aimed at constructing a multipose-specific feature model for online face recognition and tracking in a search and rescue operation. This paper proposes an integrated multipose face tracking and recognition system mounted on an unmanned aerial vehicle (UAV). The face model is constructed online by multi-scale block local binary pattern (MB-LBP) for face recognition. The generic and specific face models are further integrated for face tracking. Mechanisms for the online update of face models to retrieve loss of face tracking are also implemented. The results show that the proposed system achieves stable and robust tracking despite uncertainties (e.g. non-rigid human face, face expression changes, different face poses, complex background, varying illumination, partial visual occlusion, and pose changes). The target loss during tracking can be retrieved correctly. In face recognition, the multipose-specific face model can achieve an accuracy (above 70%). The results demonstrate the feasibility of the proof-of-concept using UAV or ground mobile robot (GMR) for real-time face recognition and tracking.