一种新型红外立体视觉定位系统研究

为实现视觉目标可靠快速三维定位,设计了一种新型的基于DSP的嵌入式主动双目红外视觉系统。与普通的红外相机不同,该系统采用常规CCD相机加红外滤光片的新方法采集红外图像,通过红外滤光后的低噪声图像可以转化为低噪声二值化图像,以此提高目标定位的准确度。提出了离散化遍历视觉目标搜索算法,通过离散控制两个相机的旋转和倾斜运动,实现视觉目标快速主动遍历搜索。实验证明,该系统目标三维定位有良好的鲁棒性。     

Stereo vision system for moving object detecting and locating based on CMOS image sensor and DSP chip

Currently, most of the stereo vision systems are constructed on PC-based or multi-CPU combination structures with two CCD cameras. It is difficult to be applied in movable plants for stand-alone requirement. Due to electronic technology development, the complementary metal-oxide semiconductor (CMOS) image sensor has been widely used in a lot of electronic commercial products and the digital signal processor (DSP) operation speed and capacity are good enough for stereo vision system requirement. Here, a new stereo vision platform is designed with TMS320C6416 DSK board integrated with two CMOS color image sensors for detecting and locating moving objects. The data communication interface, system monitoring timing flow, and image pre-processing software programs are developed, too. This system can be used to detect and track any moving object without object color and shape limitations of previous study. Experimental results are used to evaluate this system’s dynamic performance. This low cost stereo vision system can be employed in movable platform for stand-alone application, i.e., mobile robot.     

基于地面先验的3D目标检测算法

为了提高自动驾驶汽车环境感知的性能,增强单目相机对障碍物三维和边界信息的感知能力,提出了一种基于地面先验的3D目标检测算法。基于优化的中心网络(CenterNet)模型,以DLA(deep layer aggregation)为主干网络,增加目标3D边沿框中心点冗余信息预测。根据自动驾驶场景的地面先验信息,结合针孔相机模型,获取目标3D中心深度信息,以优化深度网络学习效果。使用KITTI 3D数据集评测算法性能,结果表明:在保证2D目标检测准确性的基础上,该算法运行帧率约20 fps,满足自动驾驶感知实时性要求;同时相比于CenterNet模型,在平均方位角得分(average orientation score)和鸟视图平均准确率(bird eye view AP)上分别有4.4和4.4%的性能提升。因而,该算法可以提高自动驾驶汽车对障碍物三维和边界信息的感知能力。     

结合时空一致性的FairMOT跟踪算法优化

目的 视频多目标跟踪(multiple object tracking, MOT)是计算机视觉中的一项重要任务,现有研究分别针对目标检测和目标关联部分进行改进,均忽视了多目标跟踪中的不一致问题。不一致问题主要包括3方面,即目标检测框中心与身份特征中心不一致、帧间目标响应不一致以及训练测试过程中相似度度量方式不一致。为了解决上述不一致问题,本文提出一种基于时空一致性的多目标跟踪方法,以提升跟踪的准确度。方法 从空间、时间以及特征维度对上述不一致性进行修正。对于目标检测框中心与身份特征中心不一致,针对每个目标检测框中心到特征中心之间的空间差异,在偏移后的位置上提取目标的ReID(re-identification)特征;对帧间响应不一致,使用空间相关计算相邻帧之间的运动偏移信息,基于该偏移信息对前一帧的目标响应进行变换后得到帧间一致性响应信息,然后对目标响应进行增强;对训练和测试过程中的相似度度量不一致,提出特征正交损失函数,在训练时考虑目标两两之间的相似关系。结果 在3个数据集上与现有方法进行比较。在MOT17、MOT20和Hieve数据集中,MOTA(multiple object t...     

Binocular motion tracking by gaze fixation control and three-dimensional shape reconstruction

It is an easy task for the human visual system to gaze continuously at an object moving in three-dimensional (3-D) space. While tracking the object, human vision seems able to comprehend its 3-D shape with binocular vision. We conjecture that, in the human visual system, the function of comprehending the 3-D shape is essential for robust tracking of a moving object. In order to examine this conjecture, we constructed an experimental system of binocular vision for motion tracking. The system is composed of a pair of active pan-tilt cameras and a robot arm. The cameras are for simulating the two eyes of a human while the robot arm is for simulating the motion of the human body below the neck. The two active cameras are controlled so as to fix their gaze at a particular point on an object surface. The shape of the object surface around the point is reconstructed in real-time from the two images taken by the cameras based on the differences in the image brightness. If the two cameras successfully gaze at a single point on the object surface, it is possible to reconstruct the local object shape in real-time. At the same time, the reconstructed shape is used for keeping a fixation point on the object surface for gazing, which enables robust tracking of the object. Thus these two processes, reconstruction of the 3-D shape and maintaining the fixation point, must be mutually connected and form one closed loop. We demonstrate the effectiveness of this framework for visual tracking through several experiments.     

A New Approach to Pose Detection using a Trinocular Stereovision System

Object location and tracking is a major issue in computer vision. This problem is normally solved through the extraction of representative features of the object, and the two-dimensional coordinates of these image features are used to compute the position of the object. When more than one camera is used, a certain similarity measure between the image features extracted from both stereoscopic images helps to match the correspondences. In this way, three-dimensional measurements can be recovered from the 2D coordinates of the features extracted from different cameras. In this paper the use of a trinocular system is considered to estimate both the position and velocity of known objects by using their apparent area, and with no use of the image-plane coordinates of the object 's features. A high precision low-level image processor has been developed for performing object labeling and noise filtering of the images at video rate. Then, a position measurement tool uses the apparent area captured by every camera to locate the object. This enables us to estimate the position of the object. Finally, a prediction tool refines the estimation in locating the object. We show the performance of the trinocular system with a real implementation. This system has been designed to process the images provided by any conventional of high-speed cameras at video rate.     

An efficient continuous tracking system in real-time surveillance application

A tracking object must present a proper field of view (FOV) in a multiple active camera surveillance system; its clarity can facilitate smooth processing by the surveillance system before further processing, such as face recognition. However, when pan–tilt–zoom (PTZ) cameras are used, the tracking object can be brought into the FOV by adjusting its intrinsic parameters; consequently, selection of the best-performing camera is critical. Performance is determined by the relative positions of the camera and the tracking objects, image quality, lighting and how much of the front side of the object faces the camera. In a multi-camera surveillance system, both camera hand-off and camera assignment play an important role in automated and persistent tracking, which are typical surveillance requirements. This study investigates the use of automatic methods for tracking an object across cameras in a surveillance network using PTZ cameras. An automatic, efficient continuous tracking scheme is developed. The goal is to determine the decision criteria for hand-off using Sight Quality Indication (SQI) (which includes information on the position of the object and the proportion of the front of object faces the camera), and to perform the camera handoff task in a manner that optimizes the vision effect associated with monitoring. Experimental results reveal that the proposed algorithm can be efficiently executed, and the handoff method for feasible and continuously tracking active objects under real-time surveillance.     

Efficient pose estimation using view-based object representations

We present an efficient method for estimating the pose of a three-dimensional object. Its implementation is embedded in a computer vision system which is motivated by and based on cognitive principles concerning the visual perception of three-dimensional objects. Viewpoint-invariant object recognition has been subject to controversial discussions for a long time. An important point of discussion is the nature of internal object representations. Behavioral studies with primates, which are summarized in this article, support the model of view-based object representations. We designed our computer vision system according to these findings and demonstrate that very precise estimations of the poses of real-world objects are possible even if only a small number of sample views of an object is available. The system can be used for a variety of applications.     

M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene

When occlusion is minimal, a single camera is generally sufficient to detect and track objects. However, when the density of objects is high, the resulting occlusion and lack of visibility suggests the use of multiple cameras and collaboration between them so that an object is detected using information available from all the cameras in the scene.In this paper, we present a system that is capable of segmenting, detecting and tracking multiple people in a cluttered scene using multiple synchronized surveillance cameras located far from each other. The system is fully automatic, and takes decisions about object detection and tracking using evidence collected from many pairs of cameras. Innovations that help us tackle the problem include a region-based stereo algorithm capable of finding 3D points inside an object knowing only the projections of the object (as a whole) in two views, a segmentation algorithm using bayesian classification and the use of occlusion analysis to combine evidence from different camera pairs.The system has been tested using different densities of people in the scene. This helps us determine the number of cameras required for a particular density of people. Experiments have also been conducted to verify and quantify the efficacy of the occlusion analysis scheme.     

Motion extrapolation of auditory–visual targets

Many tasks involve the precise estimation of speed and position of moving objects, for instance to catch or avoid objects that cohabit in our environment. Many of these objects are characterised by signal representations in more than one modality, such as hearing and vision. The aim of this study was to investigate the extent to which the simultaneous presentation of auditory and visual signals enhances the estimation of motion speed and instantaneous position. Observers are asked to estimate the instant when a moving object arrives at a target spatial position by pressing a response button. This task requires observers to estimate the speed of the moving object and to calibrate the timing of their manual response such that it coincides with the true arrival time of the moving object. When both visual and auditory motion signals are available, the variability in estimating the arrival time of the moving object is significantly reduced compared to the variability in the unimodal conditions. This reduction in variability is consistent with optimal integration of the auditory and visual speed signals. The average bias in the estimated arrival times depends on the motion speed: for medium speeds (17 deg/s) observers’ subjective arrival times are earlier than the true arrival times; for high speeds (47 deg/s) observers exhibit a (much smaller) bias in the other direction. This speed-dependency suggests that the bias is due to an error in estimating the motion speeds rather than an error in calibrating the timing of the motor response. Finally, in this temporal localization task, the bias and variability show similar patterns for motion defined by vision, audition or both.     

Visually guided grasping in unstructured environments

We present simple and robust algorithms which combine uncalibrated stereo vision and a robot manipulator to enable it locate, reach and grasp unmodelled objects in unstructured environments. In the first stage, an operator indicates the object to be grasped by simply pointing at it. Next, the vision system segments the indicated object from the background, and plans a suitable grasp strategy. Finally, the robotic arm reaches out towards the object and executes the grasp. Uncalibrated stereo vision allows the system to continue to operate in the presence of errors in the kinematics of the robot manipulator and unknown changes in the position, orientation and intrinsic parameters of the stereo cameras during operation.     

基于自适应分割和立体视觉的快速障碍检测

首先实时提取参考路面的平均色度值,对图像进行自适应分割,初步划分出可疑障碍区域;然后选取可疑障碍区域的对角线,利用立体视觉的方法进一步检测出障碍区域.证明了两摄像机平行放置时左右图像中区域对角线的对应关系,在此基础上进行匹配计算.该方法能适应光照条件变化,提高立体视觉算法速度,实验结果表明了该方法的有效性和可靠性.     

Trinocular vision for AGV guidance: Path localization and obstacle detection

This paper presents an integrated approach to the problem of AGV guidance using trinocular vision. Although this method uses three cameras, we will not do any processings related to stereo matching nor 3-D reconstruction. Instead, our trinocular vision is divided into two subsystems: monocular vision for path localization and binocular vision for obstacle detection. First, we demonstrate the geometric principle of 2-D vision. Then, we show its application for the task of AGV's path localization as well as the task of AGV's obstacle detection in the framework of trinocular vision. Experimental results with real images prove the usefulness of our method.     

An algorithm for vertex detection

Optical pattern recognition is playing an ever increasing role in the automation of manufacturing processes. Machine vision systems have emerged as an alternative for part qualification (the optical pattern recognition of objects) in manufacturing system. Routine tasks of the vision system are those of identification and determination of position and angular orientation in relation to reference coordinates.A PC based vision system was used to acquire the 2D image of flat parts (of polygonal geometry). The edges of this image were identified using a boundary encoding technique. Valid intersections of the edges were defined by using a bounding box, to decipher the co-ordinates of the vertices. Thus the identification of the object was done totally in software.     

监控系统中的多摄像头协同算法

给出了一个分布式多摄像头监控系统结构,描述了一种使用路径模型和目标外形变化估计的数据融合方案,提出了一种新的在分布式监控系统中的多摄像头协同算法(PDA,PriorityandDistanceAlgorithm)。提出的算法基于多摄像头数据融合结果,根据任务优先级、目标与摄像头之间的距离及目标的可见性情况,分配摄像头给目标,其特点是使系统中有高优先级并距离摄像头最近的可见目标优先分配摄像头。实验结果表明提出算法能协同多摄像头可靠地跟踪人。     

Systems and algorithms for autonomous and scalable crowd surveillance using robotic PTZ cameras assisted by a wide-angle camera

We report an autonomous surveillance system with multiple pan-tilt-zoom (PTZ) cameras assisted by a fixed wide-angle camera. The wide-angle camera provides large but low resolution coverage and detects and tracks all moving objects in the scene. Based on the output of the wide-angle camera, the system generates spatiotemporal observation requests for each moving object, which are candidates for close-up views using PTZ cameras. Due to the fact that there are usually much more objects than the number of PTZ cameras, the system first assigns a subset of the requests/objects to each PTZ camera. The PTZ cameras then select the parameter settings that best satisfy the assigned competing requests to provide high resolution views of the moving objects. We propose an approximation algorithm to solve the request assignment and the camera parameter selection problems in real time. The effectiveness of the proposed system is validated in both simulation and physical experiment. In comparison with an existing work using simulation, it shows that in heavy traffic scenarios, our algorithm increases the number of observed objects by over 210%.