Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol

Common benchmark data sets, standardized performance metrics, and baseline algorithms have demonstrated considerable impact on research and development in a variety of application domains. These resources provide both consumers and developers of technology with a common framework to objectively compare the performance of different algorithms and algorithmic improvements. In this paper, we present such a framework for evaluating object detection and tracking in video: specifically for face, text, and vehicle objects. This framework includes the source video data, ground-truth annotations (along with guidelines for annotation), performance metrics, evaluation protocols, and tools including scoring software and baseline algorithms. For each detection and tracking task and supported domain, we developed a 50-clip training set and a 50-clip test set. Each data clip is approximately 2.5 minutes long and has been completely spatially/temporally annotated at the I-frame level. Each task/domain, therefore, has an associated annotated corpus of approximately 450,000 frames. The scope of such annotation is unprecedented and was designed to begin to support the necessary quantities of data for robust machine learning approaches, as well as a statistically significant comparison of the performance of algorithms. The goal of this work was to systematically address the challenges of object detection and tracking through a common evaluation framework that permits a meaningful objective comparison of techniques, provides the research community with sufficient data for the exploration of automatic modeling techniques, encourages the incorporation of objective evaluation into the development process, and contributes useful lasting resources of a scale and magnitude that will prove to be extremely useful to the computer vision research community for years to come.     

Multiparameter method for analysis and selection of motion estimation algorithms for video compression

This paper proposes a new multiparameter method for analysis and selection of motion estimation algorithms for video compression. We present motion estimation algorithms, results of computer simulations and illustrate the analysis with tables, PSNR and performance plots. Numerous algorithms and tests for analysis of algorithm performance for video compression have recently been suggested, which has resulted in a need for effective evaluation methods. A highly qualified expert is also needed to evaluate the test results. The more input parameters used the more complex and subjective the evaluation will be. Our multiparameter method for algorithm analysis and selection eliminates subjectivity and provides a qualitative and quantitative evaluation of the tested algorithms for any number of algorithms and parameters. We propose two new methods of evaluation: (1) a quality method—a graphic method using the Pareto approach, and (2) a quantity method which obtains an integrated parameter composed of numerous evaluation parameters. In addition, we evaluate various motion estimation algorithms accordingly to two different implementation strategies: (a) using a software video encoder that depends on available processing resources using a computational complexity–rate–distortion (C–R–D) evaluation framework and (b) using a power-limited video encoder implemented on mobile or handheld computing platform by using energy–rate–distortion (E–R–D) behavior.

Agent-based reconfigurable architecture for real-time object tracking

Most conventional object tracking algorithms are implemented on general-purpose processors in software due to its great flexibility. However, the real-time performance is hard to achieve due to the inherent characteristics of the sequential processing of these processors. To tackle this issue, a reconfigurable system-on-chip (rSoC) platform with microprocessors and FPGAs is applied in this paper. To simplify the hardware/software interface, a Belief–Desire–Intention (BDI)-based multi-agent architecture is proposed as the unified framework. Then an agent-based task graph and two heuristic partitioning methods are proposed to partition the hardware and software on an rSoC platform. Compared to the module-based architecture, this BDI-based multi-agent architecture provides more efficiency, flexibility, autonomy, and scalability for the real-time tracking systems. A particle swarm optimization (PSO)-based object detection and tracking algorithm is applied to evaluate the proposed architecture. Extensive experimental results of object tracking demonstrate that the proposed architecture is efficient and highly robust with real-time performance.     

Evaluation of key frame-based retrieval techniques for video

We investigate the application of a variety of content-based image retrieval techniques to the problem of video retrieval. We generate large numbers of features for each of the key frames selected by a highly effective shot boundary detection algorithm to facilitate a query by example type search. The retrieval performance of two learning methods, boosting and k-nearest neighbours, is compared against a vector space model. We carry out a novel and extensive evaluation to demonstrate and compare the usefulness of these algorithms for video retrieval tasks using a carefully created test collection of over 6000 still images, where performance is measured against relevance judgements based on human image annotations. Three types of experiment are carried out: classification tasks, category searches (both related to automated annotation and summarisation of video material) and real world searches (for navigation and entry point finding). We also show graphical results of real video search tasks using the algorithms, which have not previously been applied to video material in this way.     

数据驱动的智能车辆目标检测能力测试评价方法

本文针对智能车辆目标检测能力测评存在的指标体系不完整、量化程度和测评实时性低等问题,聚焦智能车辆目标检测能力中的目标分类和目标识别,在这两个测评项目上提出了一套量化的评价指标体系,并用TOPSIS方法进行综合的评价.然后在此指标体系的基础上搭建数据驱动的智能车辆目标检测能力测评平台,平台可满足对智能车辆目标检测能力测评的实时性要求.最后采用了若干组车辆检测算法对指标体系进行验证.     

Modified intelligent scissors and adaptive frame skipping for video object segmentation

MPEG-4 introduces the concept of video object to support content-based functionalities. Video object segmentation is a crucial step for object-based coding and manipulation. In this paper, a robust semi- automatic video object segmentation scheme is proposed. To efficiently and accurately define the initial object contour, modified intelligent scissors is proposed on the basis of original intelligent scissors. It can improve about 6–8 times the processing speed with only slight sacrifice of accuracy, which just meets the requirements of initial object extraction for semi-automatic approach. To avoid errors accumulating and propagating during object tracking, an adaptive frame skipping scheme is proposed to decompose video sequence into video clips. For rigid and non-rigid video objects, two different image segmentation algorithms are utilized, and then region-based backward projection technique is adopted to interpolate the video object plane (VOPs) of other frames within every video clip. The proposed approach can cope with occlusion/disocclusion problem to most extent. Experimental results demonstrate the effectiveness and robustness of the method.     

Spatial Color Indexing and Applications

We define a new image feature called the color correlogram and use it for image indexing and comparison. This feature distills the spatial correlation of colors and when computed efficiently, turns out to be both effective and inexpensive for content-based image retrieval. The correlogram is robust in tolerating large changes in appearance and shape caused by changes in viewing position, camera zoom, etc. Experimental evidence shows that this new feature outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval. We also provide a technique to cut down the storage requirement of the correlogram so that it is the same as that of histograms, with only negligible performance penalty compared to the original correlogram.We also suggest the use of color correlogram as a generic indexing tool to tackle various problems arising from image retrieval and video browsing. We adapt the correlogram to handle the problems of image subregion querying, object localization, object tracking, and cut detection. Experimental results again suggest that the color correlogram is more effective than the histogram for these applications, with insignificant additional storage or processing cost.     

基于MPC8260的视频会议终端设计与实现

设计了一套多媒体视频会议终端,系统硬件平台以MPC8260为控制核心,DM642和SM712为音/视频处理显示模块,采用PCI总线进行互连。软件部分以VxWorks为操作系统平台,SIP协议栈为系统通信软交换协议,实现多媒体语音/视频通信。系统组网应用结果表明本系统软、硬件协同性好,具有较高的实时性和稳定性。     

基于OpenCL与FPGA异构模式的Sobel算法研究

随着计算机科学技术的迅速发展,嵌入式领域实时图像处理应用越来越广泛,然而传统硬件因为自身架构导致并行化程度不高,针对在视频监控、机器视觉、视频压缩、医疗影像分析等领域需要对图像进行高性能计算的问题,提出一种以OpenCL软件模型和FPGA异构模式的高性能图像处理解决方案,实现了图像显示和OpenCL加速功能,以Sobel边缘检测算法为研究对象,进行了算法并行性分析,并在系统中运用OpenCL加速内核算法,与基本的ARM平台和OpenCL共享内存加速机制相比较,展开性能测试,对加速效果进行了研究。实验数据表明,使用该系统处理不同分辨率的图像,OpenCL加速子系统的处理较基于片上ARM硬核的软件处理,实现相同功能上有100倍左右的性能提升。     

使用修改的豪氏道夫距离自动提取运动对象

新的视音频编码标准MPEG－4增加了支持基于内容的功能,它把视频序列分割成语义意义上的视频对象（VO）视频对象在某一瞬时的：“快照”称为视频对象平面（VOP）,且一系列VOP表示一个运动对象,VOP分割相当困难,这主要是因为物理对象通常不以亮度,彩色或光流等低级特征来表达,所以经典的分割方法无法获得有意义的分割结果,为了对这种视频运动图象进行有效的提取,提出了一种基于修改的豪氏道夫对象踊跃器的自动VOP分割方法,首先提取出初始模型,然后用跟踪器在序列中继帧中跟踪此对象,再对模型逐帧修改,以适应对象在后继帧中形状的旋转和变化,最后根据一系列二值模型来提取出视频对象,此外,为了提高分割效果帮减少复杂性,还使用了静 背景滤除技术来滤除静态背景,实验结果表明,该算法是有效的。     

RM及其扩展可调度性判定算法性能分析

可调度性判定是实时调度算法的关键问题·单调速率算法RM(ratemonotonic)及其扩展是应用广泛的实时调度算法,大量文献讨论了实时任务在这些算法下的可调度性判定,给出了相应的判定算法·但迄今为止,对这些判定算法的性能分析都是理论上的定性分析或者只是少数几种判定算法之间的简单比较,这不利于实时系统的开发·归纳了RM及其扩展的可调度性判定算法,通过测试平台,系统地测试和分析了各算法的性能和适用场合,讨论了各种条件和实现方式对算法性能和可调度性的影响·     

Face Detection: A Survey

In this paper we present a comprehensive and critical survey of face detection algorithms. Face detection is a necessary first-step in face recognition systems, with the purpose of localizing and extracting the face region from the background. It also has several applications in areas such as content-based image retrieval, video coding, video conferencing, crowd surveillance, and intelligent human–computer interfaces. However, it was not until recently that the face detection problem received considerable attention among researchers. The human face is a dynamic object and has a high degree of variability in its apperance, which makes face detection a difficult problem in computer vision. A wide variety of techniques have been proposed, ranging from simple edge-based algorithms to composite high-level approaches utilizing advanced pattern recognition methods. The algorithms presented in this paper are classified as either feature-based or image-based and are discussed in terms of their technical approach and performance. Due to the lack of standardized tests, we do not provide a comprehensive comparative evaluation, but in cases where results are reported on common datasets, comparisons are presented. We also give a presentation of some proposed applications and possible application areas.     

基于TMS320DM642芯片视频系统的设计与应用

由于环境以及其它因素的影响,视频系统的成像常常存在许多不足.为了弥补这类不足,在阐述了基于TMS320DM642视频系统设计原理及实现方法的基础上,提出了从硬件架构到软件设计的整体方案;并结合SAA7105＆SAA7115视频编、解码芯片以及外部存储器,实现了一个高性能的实时图像处理系统.结合实际项目简述了系统的应用,使其具有良好的应用前景和市场价值.     

智能视频算法评估综述

智能视频监控在公共安全和民用领域已经显现出有效与广阔的应用前景,备受国内外学者和众多实际工程领域专家的高度关注,制定有效、公正的智能视频算法评估准则是其应用过程中的一个重要环节.文中对当前国际上智能视频算法的评估工作进行了比较全面的综述,重点是运动检测、目标跟踪的性能评估.最后指出了智能视频算法评估的发展方向.     

Automatic text segmentation and text recognition for video indexing

Efficient indexing and retrieval of digital video is an important function of video databases. One powerful index for retrieval is the text appearing in them. It enables content-based browsing. We present our new methods for automatic segmentation of text in digital videos. The algorithms we propose make use of typical characteristics of text in videos in order to enable and enhance segmentation performance. The unique features of our approach are the tracking of characters and words over their complete duration of occurrence in a video and the integration of the multiple bitmaps of a character over time into a single bitmap. The output of the text segmentation step is then directly passed to a standard OCR software package in order to translate the segmented text into ASCII. Also, a straightforward indexing and retrieval scheme is introduced. It is used in the experiments to demonstrate that the proposed text segmentation algorithms together with existing text recognition algorithms are suitable for indexing and retrieval of relevant video sequences in and from a video database. Our experimental results are very encouraging and suggest that these algorithms can be used in video retrieval applications as well as to recognize higher level semantics in videos.