A Visual Global Positioning System for Unmanned Aerial Vehicles Used in Photogrammetric Applications

The combination of photogrammetric aerial and terrestrial recording methods can provide new opportunities for photogrammetric applications. A UAV (Unmanned Aerial Vehicle), in our case a helicopter system, can cover both the aerial and quasi-terrestrial image acquisition methods. A UAV can be equipped with an on-board high resolution camera and a priori knowledge of the operating area where to perform photogrammetric tasks. In this general scenario our paper proposes vision-based techniques for localizing a UAV. Only natural landmarks provided by a feature tracking algorithm will be considered, without the help of visual beacons or landmarks with known positions. The novel idea is to perform global localization, position tracking and localization failure recovery (kidnapping) based only on visual matching between current view and available georeferenced satellite images. The matching is based on SIFT features and the system estimates the position of the UAV and its altitude on the base of the reference image. The vision system replaces the GPS signal combining position information from visual odometry and georeferenced imagery. Georeferenced satellite or aerial images must be available on-board beforehand or downloaded during the flight. The growing availability of high resolution satellite images (e.g., provided by Google Earth or other local information sources) makes this topic very interesting and timely. Experiments with both synthetic (i.e., taken from satellites or datasets and pre elaborated) and real world images have been performed to test the accuracy and the robustness of our method. Results show sufficient performance if compared with common GPS systems and give a good performance also in the altitude estimation, even if in this last case there are only preliminary results.     

面向远程光体积描记的人脸检测与跟踪

目的 远程光体积描记（remote photoplethysmograph,rPPG）是一种基于视频的非接触心率测量方法,通过跟踪人脸皮肤区域并从中提取周期性微弱变化的颜色信号估计出心率。目前基于级联回归树的人脸地标方法训练的Dlib库,由于能快速准确定位人脸轮廓,正逐渐被研究者用于跟踪皮肤感兴趣区域（region of interest,ROI）。由于实际应用中存在地标无规则抖动,且现有研究没有考虑目标晃动的影响,因此颜色信号提取不准确,心率估计精度不佳。为了克服以上缺陷,提出一种基于Dlib的抗地标抖动和运动晃动的跟踪方法。方法 本文方法主要包含两个步骤：首先,通过阈值判断两帧间地标的区别,若近似则沿用地标,反之使用当前帧地标以解决抖动问题。其次,针对运动晃动,通过左右眼地标中点计算旋转角度,矫正晃动的人脸,保证ROI在运动中也能保持一致。结果 通过信噪比（signal-to-noise,SNR）、平均绝对误差（mean absolute error,MAE）和均方根误差（root mean squared error,RMSE）来评价跟踪方法在rPPG中的测量表现。经在UBFC-RPPG（stands for Univ.Bourgogne Franche-Comté Remote PhotoPlethysmoGraphy）和PURE（Pulse Rate Detection Dataset）数据集测试,与Dlib相比,本文方法rPPG测量结果在UBFC-RPPG中SNR提高了约0.425 dB,MAE提高0.291 5 bpm,RMSE降低0.645 3 bpm;在PURE中SNR降低了0.041 1 dB,MAE降低0.065 2 bpm,RMSE降低0.271 8 bpm。结论 本文方法相比于Dlib有效提高跟踪框稳定性,在静止和运动中都能跟踪相同ROI,适合rPPG应用。     

Embedded omni-vision navigator based on multi-object tracking

This paper presents an embedded omni-vision navigation system which involves landmark recognition, multi-object tracking, and vehicle localization. A new tracking algorithm, the feature matching embedded particle filter, is proposed. Landmark recognition is used to provide the front-end targets. A global localization method for omni-vision based on coordinate transformation is also proposed. The digital signal processor (DSP) provides a hardware platform for on-board tracker. Dynamic navigator employs DSP tracker to follow the landmarks in real time during the arbitrary movement of the vehicle and computes the position for localization based on time sequence images analysis. Experimental results demonstrated that the navigator can efficiently offer the vehicle guidance.     

Human skeleton tracking from depth data using geodesic distances and optical flow

In this paper, we present a method for human full-body pose estimation from depth data that can be obtained using Time of Flight (ToF) cameras or the Kinect device. Our approach consists of robustly detecting anatomical landmarks in the 3D data and fitting a skeleton body model using constrained inverse kinematics. Instead of relying on appearance-based features for interest point detection that can vary strongly with illumination and pose changes, we build upon a graph-based representation of the depth data that allows us to measure geodesic distances between body parts. As these distances do not change with body movement, we are able to localize anatomical landmarks independent of pose. For differentiation of body parts that occlude each other, we employ motion information, obtained from the optical flow between subsequent intensity images. We provide a qualitative and quantitative evaluation of our pose tracking method on ToF and Kinect sequences containing movements of varying complexity.     

基于全景视觉的机器人回航方法

提出一种使用全景视觉系统引导机器人回航的方法.利用全景视觉装置采集出发位置(Home位置)的全景图像,使用SURF(Speeded-Up Robust Feature)算法提取全景图像中的特征点作为自然路标点.机器人回航过程中,将当前位置获得的全景图像与Home位置的全景图像进行特征匹配,确定白然路标点之间的对应关系....     

Vision-Based Odometry and SLAM for Medium and High Altitude Flying UAVs

This paper proposes vision-based techniques for localizing an unmanned aerial vehicle (UAV) by means of an on-board camera. Only natural landmarks provided by a feature tracking algorithm will be considered, without the help of visual beacons or landmarks with known positions. First, it is described a monocular visual odometer which could be used as a backup system when the accuracy of GPS is reduced to critical levels. Homography-based techniques are used to compute the UAV relative translation and rotation by means of the images gathered by an onboard camera. The analysis of the problem takes into account the stochastic nature of the estimation and practical implementation issues. The visual odometer is then integrated into a simultaneous localization and mapping (SLAM) scheme in order to reduce the impact of cumulative errors in odometry-based position estimation approaches. Novel prediction and landmark initialization for SLAM in UAVs are presented. The paper is supported by an extensive experimental work where the proposed algorithms have been tested and validated using real UAVs.     

Mobile Camera Localization Using Apollonius Circles and Virtual Landmarks

We investigate the localization of a camera subject to a planar motion with horizontal optical axis in the presence of known vertical landmarks. Under these assumptions, a calibrated camera can measure the distance to the viewed landmarks. We propose to replace the trilateration method by intersecting a pair of Chasles-Apollonius circles. In the case of square pixels but unknown focal length we introduce a new method to recover the camera position from one image with three vertical landmarks. To this end we consider virtual landmarks and Apollonius-like circles. We extend this method in order to deal with an unknown principal point by using four landmarks.     

Robot navigation using panoramic tracking

A vision-based navigation system is presented for determining a mobile robot's position and orientation using panoramic imagery. Omni-directional sensors are useful in obtaining a 360° field of view, permitting various objects in the vicinity of a robot to be imaged simultaneously. Recognizing landmarks in a panoramic image from an a priori model of distinct features in an environment allows a robot's location information to be updated. A system is shown for tracking vertex and line features for omni-directional cameras constructed with catadioptric (containing both mirrors and lenses) optics. With the aid of the panoramic Hough transform, line features can be tracked without restricting the mirror geometry so that it satisfies the single viewpoint criteria. This allows the use of rectangular scene features to be used as landmarks. Two paradigms for localization are explored, with experiments conducted with synthetic and real images. A working implementation on a mobile robot is also shown.     

Saliency and spatial information-based landmark selection for mobile robot navigation in natural environments

In this paper, a landmark selection and tracking approach is presented for mobile robot navigation in natural environments, using textural distinctiveness-based saliency detection and spatial information acquired from stereo data. The presented method focuses on achieving high robustness of tracking rather than self-positioning accuracy. The landmark selection method is designed to select a small amount of the most salient feature points in a wide variety of sparse unknown environments to ensure successful matching. Landmarks are selected by an iterative algorithm from a textural distinctiveness-based saliency map extended with spatial information, where a repulsive potential field is created around the position of each already selected landmark for better distribution in order to increase robustness. The template matching of landmarks is aided with visual odometry-based motion estimation. Other robustness increasing strategies includes estimating landmark positions by unscented Kalman filters as well as from surrounding landmarks. Experimental results show that the introduced method is robust and suitable for natural environments.     

Efficient generic face model fitting to images and videos

In this paper we present a robust and lightweight method for the automatic fitting of deformable 3D face models on facial images. Popular fitting techniques such as those based on statistical models of shape and appearance require a training stage based on a set of facial images and their corresponding facial landmarks, which have to be manually labeled. Therefore, new images in which to fit the model cannot differ too much in shape and appearance (including illumination variation, facial hair, wrinkles, etc.) from those used for training. By contrast, our approach can fit a generic face model in two steps: (1) the detection of facial features based on local image gradient analysis and (2) the backprojection of a deformable 3D face model through the optimization of its deformation parameters. The proposed approach can retain the advantages of both learning-free and learning-based approaches. Thus, we can estimate the position, orientation, shape and actions of faces, and initialize user-specific face tracking approaches, such as Online Appearance Models (OAMs), which have shown to be more robust than generic user tracking approaches. Experimental results show that our method outperforms other fitting alternatives under challenging illumination conditions and with a computational cost that allows its implementation in devices with low hardware specifications, such as smartphones and tablets. Our proposed approach lends itself nicely to many frameworks addressing semantic inference in face images and videos.     

基于粒子滤波的未知环境下机器人同时定位、地图构建与目标跟踪

为了解决机器人在未知环境下的目标跟踪问题,提出了一种基于粒子滤波的机器人同时定位、地图构建与目标跟踪方法．该方法采用Rao-Blackwellized粒子滤波器对机器人位姿状态、标志柱分布和目标位置同时进行估计．该方法中,粒子群的总体分布情况表征机器人位姿状态,而每个粒子均包含2类EKF滤波器,其中一类用来完成对标志柱分布的估计,另一类用来完成对目标状态的估计,粒子的权值则由粒子状态相对于标志柱和目标状态2类相似度共同产生．通过仿真和实体机器人实验验证了该方法的有效性．     

Tracking servo method using reflective optical filter for holographic data storage system

Holographic data storage system is one of the next-generation data storage systems and is characterized by its high storage density and fast data transfer rate. However, holographic data storage systems are very sensitive to disturbances that affect the position of the media. Therefore, tracking servo control is needed to ensure a good performance of the system even if disturbances occur, such as eccentricity of the disk and external shock. In our previous researches on tracking servo methods, we used additional beams or recorded servo track images with data pages to record additional gratings with data pages. Therefore, the recording density may be reduced and the system may be complicated. In addition, the performance of the system may be compromised by cross-talk noise caused between the reference beams and additional beams. In this paper, we propose a tracking servo method using the residual beam, which is reflected by the reflective optical filter. This method does not require recording supplementary gratings or use additional beams, and only needs to record data pages. The residual beam is retrieved with desired retrieved beam by the reference beam and wasted during the retrieving process. We first constructed a holographic data storage system and designed a reflective optical filter to detect tracking error signals. After detecting the tracking error signals, a tracking servo controller using a lead-lag compensator was incorporated to reduce the tracking error signals. The performance of the designed controller was verified by simulated and experimental results. Finally, the performance of the tracking servo method was investigated by comparing the retrieved data images.     

Probabilistic relaxation method of matching visual landmarks for controlling the accurate positioning of mobile robots

The navigation problem of controlling the accurate positioning of a mobile robot (MR) with a computer vision system under conditions of uncertainty of information about its position is considered. Visual servo control is used, in which the error signal is calculated as the difference in the coordinates of natural visual landmarks detected by the computer vision system on the current and reference (received in a target position) images. To compute the error signal, a probabilistic relaxation method of correct matching of the landmarks extracted in the image is proposed. The efficiency of the proposed method has been confirmed by numerical experiments for processing real images.     

Semi-supervised facial landmark annotation

Landmark annotation for training images is essential for many learning tasks in computer vision, such as object detection, tracking, and alignment. Image annotation is typically conducted manually, which is both labor-intensive and error-prone. To improve this process, this paper proposes a new approach to estimating the locations of a set of landmarks for a large image ensemble using manually annotated landmarks for only a small number of images in the ensemble. Our approach, named semi-supervised least-squares congealing, aims to minimize an objective function defined on both annotated and unannotated images. A shape model is learned online to constrain the landmark configuration. We employ an iterative coarse-to-fine patch-based scheme together with a greedy patch selection strategy for landmark location estimation. Extensive experiments on facial images show that our approach can reliably and accurately annotate landmarks for a large image ensemble starting with a small number of manually annotated images, under several challenging scenarios.     

Vision-Based Navigation for an Electric Wheelchair Using Ceiling Light Landmark

This paper presents an autonomous wheelchair system with the capability of self-location and obstacle avoidance. The wheelchair is equipped with two TV cameras that are used for self-location and obstacle avoidance, respectively. In this system, the fluorescent ceiling lights are chosen as landmarks since they can be easily detected and do not require an additional installation. A recognition procedure of landmarks is described by which the desired landmark images in the navigational environment can be retrieved. A self-location technique using ceiling light landmarks is proposed. Using this self-location function, the wheelchair can locate itself in a world coordinate system. The path planning based on landmark and the method of generating a control scheme are presented so that the wheelchair is capable of navigating along any path from start position to goal position. A low cost and high-speed vision system for the detection and avoidance of obstacle is developed and the principal of obstacle avoidance is introduced. A number of navigation experiments are conducted for the wheelchair in an indoor environment. The experimental results indicate the effectiveness of the wheelchair system.     

The integration of optical and magnetic tracking for multi-user augmented reality

Multi-user augmented reality systems are especially dependent on precise registration. In this paper, we present a hybrid tracking system that combines optical and magnetic tracking. The magnetic tracking is used to give a robust estimate of position and orientation which then can be refined in realtime by optical tracking. The system is more precise than a magnetic tracker and both faster and more reliable than an optical tracker.     

基于最小平方中值定理的立体视觉里程计

提出了一种基于最小平方中值定理（LMedS）的立体视觉里程计方法。利用图像中尺度不变的SIFT特征点作为路标,基于KD树的最邻近点搜索算法来实现左右图像对特征点的匹配和前后帧间特征点跟踪。通过特征点的三维重建,基于最小平方中值定理估计出机器人的运动距离和方向信息。实验表明该方法在不同图像间匹配、三维路标跟踪和机器人运动估计中具有很强的鲁棒性。     

Cleaning robot navigation using panoramic views and particle clouds as landmarks

The paper describes a visual method for the navigation of autonomous floor-cleaning robots. The method constructs a topological map with metrical information where place nodes are characterized by panoramic images and by particle clouds representing position estimates. Current image and position estimate of the robot are interrelated to landmark images and position estimates stored in the map nodes through a holistic visual homing method which provides bearing and orientation estimates. Based on these estimates, a position estimate of the robot is updated by a particle filter. The robot’s position estimates are used to guide the robot along parallel, meandering lanes and are also assigned to newly created map nodes which later serve as landmarks. Computer simulations and robot experiments confirm that the robot position estimate obtained by this method is sufficiently accurate to keep the robot on parallel lanes, even in the presence of large random and systematic odometry errors. This ensures an efficient cleaning behavior with almost complete coverage of a rectangular area and only small repeated coverage. Furthermore, the topological-metrical map can be used to completely cover rooms or apartments by multiple meander parts.     

Reducing the complexity of visual navigation: Optical track controller for long‐range unmanned aerial vehicles

Using vision for navigation of airborne systems provides an opportunity for motion relative to the ground to be controlled in the absence of other supporting sensors including global navigation satellite systems. Rather than relying on computationally intensive localization techniques such as online map construction, identification and tracking of landmarks, or otherwise producing an explicit quantitative estimate of position, we propose and have experimentally demonstrated a closed‐loop visual navigation reflex which we term the optical ground course controller. The behavior is applicable to fixed wing aircraft traversing long ranges, and reduces the online computation and sensors required compared to other visual methods. This method combines the kinematics of fixed wing aircraft flight, the direction of apparent motion of an image sequence, and a magnetic compass to create a bioinspired optomotor reflex similar to those observed in insects. This behavior accurately controls track in the inertial reference frame (path taken over the ground) with only limited dependence on altitude, speed, and wind. We show that the proposed behavior is naturally convergent and stable, and present experimental results from simulation and real‐world flight demonstrating that the method performs robustly, producing improvement over both magnetic‐referenced and visual odometry‐based navigation within the limits of the sensor.     

Attitude Determination of Multirotor Aerial Vehicles Using Camera Vector Measurements

The employment of embedded cameras in navigation and guidance of Unmanned Aerial Vehicles (UAV) has attracted the focus of many academic researches. In particular, for the multirotor UAV, the camera is widely employed for applications performed in indoor environments, where the GNSS signal is often unreliable and electromagnetic interference can be a concern. In the literature, images are mostly adopted for position and velocity estimation, rather than attitude estimation. This paper proposes an attitude determination method for multirotor aerial vehicles using pairs of vector measurements taken from a downward-facing strapdown camera. The method is composed of three modules. The first one detects and identifies the visible landmarks by processing the images. The second module computes the vector measurements related to the direction from the camera to the landmarks. The third module estimates attitude from the vector measurements. In the last module, a version of the Multiplicative Extended Kalman Filter (MEKF) with sequential update is proposed as estimation method. The overall method is evaluated via Monte Carlo simulations, showing that it is effective in determining the vehicle’s attitude and revealing its properties.