基于微分平滑滤波的特征点跟踪

光流跟踪作为一种重要的二维运动估计技术,在运动目标检测和跟踪中有着重要的作用.L-K光流跟踪算法是一种利用Newton-Raphson梯度下降法进行图像匹配的算法,所以离散图像的偏导数的计算显得尤为重要.由于数字图像离散的方格结构导致在计算各阶偏导数时产生较大的误差.利用微分平滑滤波的方法先利用一个连续的基础函数模拟离...     

500-fps face tracking system

In this paper, we propose a high-speed vision system that can be applied to real-time face tracking at 500 fps using GPU acceleration of a boosting-based face tracking algorithm. By assuming a small image displacement between frames, which is a property of high-frame rate vision, we develop an improved boosting-based face tracking algorithm for fast face tracking by enhancing the Viola–Jones face detector. In the improved algorithm, face detection can be efficiently accelerated by reducing the number of window searches for Haar-like features, and the tracked face pattern can be localized pixel-wise even when the window is sparsely scanned for a larger face pattern by introducing skin color extraction in the boosting-based face detector. The improved boosting-based face tracking algorithm is implemented on a GPU-based high-speed vision platform, and face tracking can be executed in real time at 500 fps for an 8-bit color image of 512 × 512 pixels. In order to verify the effectiveness of the developed face tracking system, we install it on a two-axis mechanical active vision system and perform several experiments for tracking face patterns.     

基于并行跟踪检测框架与深度学习的目标跟踪算法

在空地协同背景下，地面目标的移动导致其在无人机视角下外观会发生较大变化，传统算法很难满足此类场景的应用要求。针对这一问题，提出基于并行跟踪和检测（PTAD）框架与深度学习的目标检测与跟踪算法。首先，将基于卷积神经网络（CNN）的目标检测算法SSD作为PTAD的检测子处理关键帧获取目标信息并提供给跟踪子；其次，检测子与跟踪子并行处理图像帧并计算检测与跟踪结果框的重叠度及跟踪结果的置信度；最后，根据跟踪子与检测子的跟踪或检测状态来判断是否对跟踪子或检测子进行更新，并对图像帧中的目标进行实时跟踪。在无人机视角下的视频序列上开展实验研究和对比分析，结果表明所提算法的性能高于PTAD框架下最优算法，而且实时性提高了13%，验证了此算法的有效性。     

An image detection technique based on morphological edge detection and background differencing for real-time traffic analysis

The real-time vehicle detection from a traffic scene is the major process in image processing based traffic data collection and analysis techniques. The most common algorithm used for real-time vehicle detection is based on background differencing and thresholding operations. The efficiency of this method of image detection is heavily dependent on the background updating and threshold selection techniques. In this paper, a new background updating and a dynamic threshold selection technique is presented. An alternative image detection technique used in image processing is based on edge detection techniques. However, an edge detector extracts the edges of the objects of a scene irrespective of whether it belongs to the background details or the objects. Therefore, to separate these two, extra information is required. We have developed a new image detection method based on background differencing and edge detection techniques, which separates the objects from their backgrounds and works well under various lighting and weather conditions. This image detection technique together with other techniques for calculating traffic parameters e.g. counting number of vehicles, works in real-time on an 80386-based microcomputer operating at a clock speed of 33 MHz.     

Detection and tracking of single-pixel targets based on trajectory continuity

A target detection and tracking algorithm has been developed to identify single-pixel targets with unknown motion from a time sequence of highly noisy images. The algorithm is based on a target trajectory continuity theory, utilizing temporal continuity and smoothness of target trajectories in both intensity and spatial coordinates in an image plane to detect and simultaneously track multiple targets. With a unique application of the trajectory continuity theory, the algorithm presents an effective engineering solution to the small target track initiation problem in under-speficied environments where an optimum solution is not possible, and at the same time unties the constraint of straight line trajectory that most optimum algorithms require for similar tasks. The algorithm design utilizes a parallel-distributed computing architecture, which aims for real-time target detection and tracking applications.     

基于Digiclops立体视觉系统的单指指尖跟踪

提出一种复杂背景下检测单指指尖位置的方法,该方法使用Digiclops立体视觉系统采集图像,并得到手指区域的子图像。对于手指正指的情况,可迅速计算出指尖的位置;对于手指侧指的情况,在手指图像基础上,设计一种鲁棒的指尖检测算法定位出指尖的位置。实验表明,该方法对指尖位置检测准确,用该方法处理每一帧图像,可实时跟踪指尖,从而实现了基于指尖跟踪的感知用户界面系统。     

局部特定空间关系统计特征的RVIN噪声检测器

目的 随机脉冲噪声（random-valued impulse noise，RVIN）检测器将局部图像统计值（local image statistics，LIS）作为图块中心像素点是否为噪声的判断依据，但LIS的描述能力较弱，在不同程度上制约了RVIN检测器的检测正确率，影响了后续开关型降噪模块的修复效果。为此，提出了一种基于局部特定空间关系统计特征的RVIN噪声检测器。方法 以局部中心像素点的8个邻域像素对数差值排序值（rank-ordered logarithmic difference，ROLD）并结合1个最小方向对数差值（minimum orientation logarithmic difference，MOLD）共9个反映局部特定空间关系的LIS统计值构成描述中心像素点是否为RVIN的噪声感知特征矢量，并通过在大量样本图块数据上提取的RVIN噪声感知特征矢量及其对应的噪声标签作为训练对（training pairs），训练获得一个基于多层感知网络（multi-layer perception，MLP）的RVIN噪声检测器。结果 对比实验从检测正确率和实际应用效果2个方面检验所提出的RVIN检测器的有效性，分别在10幅常用图像和50幅BSD （Berkeley segmentation data）纹理图像上进行测试，并与经典的脉冲噪声降噪算法中包含的噪声检测器以及MLPNNC （MLP neural network classifier）噪声检测器相比较，以漏检数、误检数和错检总数作为评价噪声检测正确率的指标。在常用图像集上本文所提RVIN检测器的漏检数和误检数较为平衡，在错检总数上排名处于所有对比算法中的前2名，为后续的降噪模块打下了很好的基础。在BSD纹理图像集上，将本文提出的RVIN检测器和GIRAF （generic iteratively reweighted annihilating filter）算法组合构成一种RVIN噪声降噪算法（proposed-GIRAF），proposed-GIRAF算法在50幅BSD图像上的峰值信噪比（peak signal-to-noise ratio，PSNR）均值在各个噪声比例下均取得了最优结果，与排名第2的对比算法相比，提升了0.471.96 dB。实验数据表明，所提出的RVIN噪声检测器的检测正确率优于现有的检测器，与修复算法联用后即可获得一种降噪效果更佳的开关型RVIN降噪算法。结论 本文提出的RVIN噪声检测器在各个噪声比例下具有鲁棒的预测准确性，配合GIRAF算法使用后，与经典的RVIN降噪算法相比，降噪效果最佳，具有很强的实用性。     

免锚检测的行人多目标跟踪算法

针对复杂环境下行人目标因检测器漏检和频繁遮挡而导致的数据关联不正确、跟踪实时性差的问题,提出了一种基于免锚检测的多目标跟踪算法.算法采用预测目标中心点热力图的方法实现目标检测定位,改善了因锚点框回归歧义所导致的漏检问题.同时在检测模型中嵌入深度表观特征提取分支,构建联合检测与跟踪的多任务网络用于提升实时性.为解决跟踪阶...     

Developing robust vision modules for microsystems applications

In this work, several robust vision modules are developed and implemented for fully automated micromanipulation. These are autofocusing, object and end-effector detection, real-time tracking and optical system calibration modules. An image based visual servoing architecture and a path planning algorithm are also proposed based on the developed vision modules. Experimental results are provided to assess the performance of the proposed visual servoing approach in positioning and trajectory tracking tasks. Proposed path planning algorithm in conjunction with visual servoing imply successful micromanipulation tasks.     

基于运动区域检测的运动目标跟踪算法*

针对传统基于模板匹配的运动目标跟踪算法存在着计算量大、模板漂移导致跟踪失败的问题,提出了一种基于运动区域检测的运动目标跟踪算法。该算法通过采用光流法对目标运动区域进行估计,计算出光流场区域的形心,确定待匹配图相匹配范围,再用模板框在已确定区域进行模板匹配跟踪。根据某开放实验室行人录像跟踪实验表明,本算法能够有效解决模板漂移问题,提高了跟踪实时性, 实现了视频对象目标的跟踪。     

A multi-cue mean-shift target tracking approach based on fuzzified region dynamic image fusion

Traditional target tracking algorithms based on single sensor images are unstable and have low accuracy. Based on regional target detection and fuzzy region rules, a fuzzy region-based multi-sensor image fusion approach is proposed in this paper. The similarity measure weight is adapted to this dynamic image fusion algorithm, while the tracking method uses the proposed multi-cue mean-shift tracking algorithm. Three experimental results using real world image sequences are evaluated using the steady state square root mean error. The fusion and tracking experiments indicate that the proposed approach is effective and efficient when aiming at a target moving from one area to a different area, which meets the robustness and real-time requirements.     

Image Processing Using RBF like Neural Networks: A ZISC-036 Based Fully Parallel Implementation Solving Real World and Real Complexity Industrial Problems

The present article concerns neural based image processing and solutions developed for industrial problems using the ZISC-036 neuro-processor, an IBM hardware processor which implements the Restricted Coulomb Energy algorithm (RCE) and the K-Nearest Neighbor algorithm (KNN). The developed neural based techniques have been applied for image enhancement in order to restore old movies (noise reduction, focus correction, etc.), to improve digital television images, or to treat images which require adaptive processing (medical images, spatial images, special effects, etc.). We also have developed and implemented on ZISC-036 neuro-processor, a neural network based solution for visual probe mark inspection in VLSI production for the IBM Essonnes plant. The main characteristics of such systems are real-time control and high reliability in detection and classification tasks. Experimental results, validating presented concepts, have been reported showing quantitative and qualitative improvement as well as the efficiency our solutions.     

结合模糊特征检测的鲁棒核相关滤波跟踪法

针对跟踪领域内由于图像模糊而导致跟踪失败的问题,提出一种结合模糊特征检测的鲁棒核相关滤波(kernelized correlation filter, KCF)跟踪法。首先,将尺度不变特征变换(scale invariant feature transform, SIFT)描述子与局部二值模式(local binary pattern, LBP)算法结合,提取模糊图像中的特征点,并采用圆形邻域描述该特征点,以降低特征向量的维度,综合构建出模糊特征检测器。其次,设置图像清晰度阈值,若当前图像清晰度低于阈值,则启动模糊特征检测器,通过特征向量间的匹配,得出跟踪目标的位置;否则,通过传统的核相关滤波法预测目标位置。最后,在公开数据集OTB-2013和OTB-2015中的测试结果表明:与其他实验算法相比,该算法可对模糊图像中的目标进行有效跟踪且精度较高。     

基于DM642的金字塔Lucas-Kanade光流法计算速度信息

通过垂直拍摄地面的摄像头连续抓拍两帧图像,从中计算出横纵向速度信息.该设计以TMS320DM642为核心,结合TI的VLIB视频处理库,利用VLIB中的Harris角点提取算法对特征点进行提取,并利用金字塔Lucas-Kanade光流法实现对特征点的大位移跟踪,对跟踪出来的横纵向位移信息进行筛选并利用帧率与位移之间的关系计算出速度信息.然后,将该设计与在PC机上用OpenCV实现的金字塔Lucas-Kanade光流法和SURF特征点跟踪匹配法进行比较,其结果表明该设计简易可行且具有实时性好的优点.最后在此基础上简要介绍了此设计的应用前景并对设计进行了总结.     

Development of head detection and tracking systems for visual surveillance

This paper proposes a technique for the detection of head nod and shake gestures based on eye tracking and head motion decision. The eye tracking step is divided into face detection and eye location. Here, we apply a motion segmentation algorithm that examines differences in moving people’s faces. This system utilizes a Hidden Markov Model-based head detection module that carries out complete detection in the input images, followed by the eye tracking module that refines the search based on a candidate list provided by the preprocessing module. The novelty of this paper is derived from differences in real-time input images, preprocessing to remove noises (morphological operators and so on), detecting edge lines and restoration, finding the face area, and cutting the head candidate. Moreover, we adopt a K-means algorithm for finding the head region. Real-time eye tracking extracts the location of eyes from the detected face region and is performed at close to a pair of eyes. After eye tracking, the coordinates of the detected eyes are transformed into a normalized vector of x-coordinate and y-coordinate. Head nod and shake detector uses three hidden Markov models (HMMs). HMM representation of the head detection can estimate the underlying HMM states from a sequence of face images. Head nod and shake can be detected by three HMMs that are adapted by a directional vector. The directional vector represents the direction of the head movement. The vector is HMMs for determining neutral as well as head nod and shake. These techniques are implemented on images, and notable success is notified.     

Real-time single scattering inside inhomogeneous materials

In this paper we propose a novel technique to perform real-time rendering of translucent inhomogeneous materials, one of the most well-known problems of computer graphics. The developed technique is based on an adaptive volumetric point sampling, done in a preprocessing stage, which associates to each sample the optical depth for a predefined set of directions. This information is then used by a rendering algorithm that combines the object’s surface rasterization with a ray tracing algorithm, implemented on the graphics processor, to compose the final image. This approach allows us to simulate light scattering phenomena for inhomogeneous isotropic materials in real time with an arbitrary number of light sources. We tested our algorithm by comparing the produced images with the result of ray tracing and showed that the technique is effective.     

A real-time vehicle detection and a novel vehicle tracking systems for estimating and monitoring traffic flow on highways

Real-time highway traffic monitoring systems play a vital role in road traffic management, planning, and preventing frequent traffic jams, traffic rule violations, and fatal road accidents. These systems rely entirely on online traffic flow info estimated from time-dependent vehicle trajectories. Vehicle trajectories are extracted from vehicle detection and tracking data obtained by processing road-side camera images. General-purpose object detectors including Yolo, SSD, EfficientNet have been utilized extensively for real-time object detection task, but, in principle, Yolo is preferred because it provides a high frame per second (FPS) performance and robust object localization functionality. However, this algorithm’s average vehicle classification accuracy is below 57%, which is insufficient for traffic flow monitoring. This study proposes improving the vehicle classification accuracy of Yolo, and developing a novel bounding box (Bbox)-based vehicle tracking algorithm. For this purpose, a new vehicle dataset is prepared by annotating 7216 images with 123831 object patterns collected from highway videos. Nine machine learning-based classifiers and a CNN-based classifier were selected. Next, the classifiers were trained via the dataset. One out of ten classifiers with the highest accuracy was selected to combine to Yolo. This way, the classification accuracy of the Yolo-based vehicle detector was increased from 57% to 95.45%. Vehicle detector 1 (Yolo) and vehicle detector 2 (Yolo + best classifier), and the Kalman filter-based tracking as vehicle tracker 1 and the Bbox-based tracking as vehicle tracker 2 were applied to the categorical/total vehicle counting tasks on 4 highway videos. The vehicle counting results show that the vehicle counting accuracy of the developed approach (vehicle detector 2 + vehicle tracker 2) was improved by 13.25% and this method performed better than the other 3 vehicle counting systems implemented in this study.     

Road Detection and Tracking from Aerial Desert Imagery

We present a fast, robust road detection and tracking algorithm for aerial images taken from an Unmanned Aerial Vehicle. A histogram-based adaptive threshold algorithm is used to detect possible road regions in an image. A probabilistic hough transform based line segment detection combined with a clustering method is implemented to further extract the road. The proposed algorithm has been extensively tested on desert images obtained using an Unmanned Aerial Vehicle. Our results indicate that we are able to successfully and accurately detect roads in 96% of the images. We experimentally validated our algorithm on over a thousand aerial images obtained using our UAV. These images consist of straight and curved roads in various conditions with significant changes in lighting and intensity. We have also developed a road-tracking algorithm that searches a local rectangular area in successive images. Initial results are presented that shows the efficacy and the robustness of this algorithm. Using this road tracking algorithm we are able to further improve the road detection and achieve a 98% accuracy.     

基于细胞神经网络运动目标跟踪算法研究

运动目标跟踪是模式识别、图像处理、计算机视觉等领域的重要课题,它把图像处理、自动控制、信息科学有机结合起来,针对背景是静止的运动物体图像序列,提出了基于细胞神经网络移动目标跟踪,该算法大部分采用细胞神经网络结构,能够实现高效、快速的移动目标跟踪,可以满足实时需要,在实验基础上验证了该算法的有效性。     

A dynamically reconfigurable multiprocessor system for high-order bidirectional associative memory-based image recognition

A High-Order Bidirectional Associative Memory (HOBAM) based image recognition system and a dynamically reconfigurable multiprocessor system that achieves real-time response have been utilized to recognize corrupt images of human faces (faces obscured by hats, glasses, masks or slight translation and scaling effects). In addition, the HOBAM, in conjunction with edge detection techniques, has been used to recognize isolated objects within multiple-object images. Successful recognition rates have been achieved in both cases.A dynamically reconfigurable multiprocessor system and parallel software have been developed to achieve real-time response for image recognition. The system consists of Inmos transputers and cross-bar switches (IMS C004) with communication links dynamically connected by circuit switching. The use of transputers and cross-bar switches combined to form a low-cost multiprocessor system connected by a switching network is reported. Moreover, the switching network, which makes message routing unnecessary, simplifies the design of the communication in parallel software. Although the HOBAM is a fully connected network, the algorithm minimizes the amount of information that needs to be exchanged between processors using a data compression technique. The detailed design of both hardware and software are discussed, as well as the use of parallel processing to significantly increase the speed ratio. The architecture of the experimental system is a cost-effective design for an embedded system for neural network applications on computer vision.