Simultaneous active camera array focus plane estimation and occluded moving object imaging

Automatically focusing and seeing occluded moving object in cluttered and complex scene is a significant challenging task for many computer vision applications. In this paper, we present a novel synthetic aperture imaging approach to solve this problem. The unique characteristics of this work include the following: (1) To the best of our knowledge, this work is the first to simultaneously solve camera array auto focusing and occluded moving object imaging problem. (2) A unified framework is designed to achieve seamless interaction between the focusing and imaging modules. (3) In the focusing module, a local and global constraint-based optimization algorithm is presented to dynamically estimate the focus plane of the moving object. (4) In the imaging module, a novel visibility analysis based active synthetic aperture imaging approach is proposed to remove the occluder and significantly improve the quality of occluded object imaging. An active camera array system has been set up and evaluated in challenging indoor and outdoor scenes. Extensive experimental results with qualitative and quantitative analyses demonstrate the superiority of the proposed approach compared with state-of-the-art approaches.     

A novel multi-object detection method in complex scene using synthetic aperture imaging

This paper proposes a novel multi-object detection method using multiple cameras. Unlike conventional multi-camera object detection methods, our method detects multiple objects using a linear camera array. The array can stream different views of the environment and can be easily reconfigured for a scene compared with the overhead surround configuration. Using the proposed method, the synthesized results can provide not only views of significantly occluded objects but also the ability of focusing on the target while blurring objects that are not of interest. Our method does not need to reconstruct the 3D structure of the scene, can accommodate dynamic background, is able to detect objects at any depth using a new synthetic aperture imaging method based on a simple shift transformation, and can see through occluders. The experimental results show that the proposed method has a good performance and can synthesize objects located within any designated depth interval with much better clarity than that using an existing method. To our best knowledge, it is the first time that such a method using synthetic aperture imaging has been proposed and developed for multi-object detection in a complex scene with a significant occlusion at different depths.     

Partially Occluded Object Recognition Using Statistical Models

In this paper, we present a new Bayesian framework for partially occluded object recognition based on matching extracted local features on a one-to-one basis with object features. We introduce two different statistical models for occlusion: one model assumes that each feature in the model can be occluded independent of whether any other features are occluded, whereas the second model uses spatially correlated occlusion to represent the extent of occlusion. Using these models, the object recognition problem reduces to finding the object hypothesis with largest generalized likelihood. We develop fast algorithms for finding the optimal one-to-one correspondence between scene features and object features to compute the generalized likelihoods under both models. We conduct experiments illustrating the differences between the two occlusion models using different quantitative metrics. We also evaluate the recognition performance of our algorithms using examples extracted from object silhouettes and synthetic aperture radar imagery, and illustrate the performance advantages of our approach over alternative algorithms.     

Robust object tracking with occlusion handle

Occlusion is a major problem for object tracking algorithms, especially for subspace-based learning algorithms like PCA. In this paper, we introduce a novel incremental subspace (robust PCA)-based object tracking algorithm to deal with the occlusion problem. The three major contributions of our works are the introduction of robust PCA to object tracking literature, a robust PCA-based occlusion handling scheme and the revised incremental PCA algorithm. In order to handle the occlusion problem in the subspace learning algorithm framework, robust PCA algorithm is employed to select part of image pixels to compute coefficients rather than the whole image pixels as in traditional PCA algorithm, which can successfully avoid the occluded pixels and therefore obtain accurate tracking results. The occlusion handling scheme fully makes use of the merits of robust PCA and can avoid false updates in occlusion, clutter, noisy and other complex situations. Besides, the introduction of incremental PCA facilitates the subspace updating process and possesses several benefits compared with traditional R-SVD-based updating methods. The experiments show that our proposed algorithm is efficient and effective to cope with common object tracking tasks, especially with strong robustness due to the introduction of robust PCA.     

基于4目阵列的计算机视觉三维重建算法

目前计算机视觉三维重建方法因需布置和标定摄像机环形拍摄场或者需要结构光而存在应用局限性问题,且算法不稳定。为此提出一种将摄像机阵列和图像配准有机结合的4目阵列重建算法,该算法不需要结构光和现场标定摄像机。经过基于包含光照和阴影的复杂室内仿真图像的实验表明,该方法能稳定有效地进行密集点云重建,且能克服现有重建方法的应用局限性与不稳定等缺陷。     

Efficient Dense Stereo with Occlusions for New View-Synthesis by Four-State Dynamic Programming

A new algorithm is proposed for efficient stereo and novel view synthesis. Given the video streams acquired by two synchronized cameras the proposed algorithm synthesises images from a virtual camera in arbitrary position near the physical cameras. The new technique is based on an improved, dynamic-programming, stereo algorithm for efficient novel view generation. The two main contributions of this paper are: (i) a new four state matching graph for dense stereo dynamic programming, that supports accurate occlusion labelling; (ii) a compact geometric derivation for novel view synthesis by direct projection of the minimum cost surface. Furthermore, the paper presents an algorithm for the temporal maintenance of a background model to enhance the rendering of occlusions and reduce temporal artefacts (flicker); and a cost aggregation algorithm that acts directly in the three-dimensional matching cost space. The proposed algorithm has been designed to work with input images with large disparity range, a common practical situation. The enhanced occlusion handling capabilities of the new dynamic programming algorithm are evaluated against those of the most powerful state-of-the-art dynamic programming and graph-cut techniques. Four-state DP is also evaluated against the disparity-based Middlebury error metrics and its performance found to be amongst the best of the efficient algorithms. A number of examples demonstrate the robustness of four-state DP to artefacts in stereo video streams. This includes demonstrations of cyclopean view synthesis in extended conversational sequences, synthesis from a freely translating virtual camera and, finally, basic 3D scene editing.     

Sparse synthetic aperture radar imaging with optimized azimuthal aperture

To counter the problem of acquiring and processing huge amounts of data for synthetic aperture radar(SAR)using traditional sampling techniques,a method for sparse SAR imaging with an optimized azimuthal aperture is presented.The equivalence of an azimuthal match filter and synthetic array beamforming is shown so that optimization of the azimuthal sparse aperture can be converted to optimization of synthetic array beamforming.The azimuthal sparse aperture,which is composed of a middle aperture and symmetrical bilateral apertures,can be obtained by optimization algorithms(density weighting and simulated annealing algorithms,respectively).Furthermore,sparse imaging of spectrum analysis SAR based on the optimized sparse aperture is achieved by padding zeros at null samplings and using a non-uniform Taylor window.Compared with traditional sampling,this method has the advantages of reducing the amount of sampling and alleviating the computational burden with acceptable image quality.Unlike periodic sparse sampling,the proposed method exhibits no image ghosts.The results obtained from airborne measurements demonstrate the effectiveness and superiority of the proposed method.     

Dynamic scene occlusion culling

Large, complex 3D scenes are best rendered in an output-sensitive way, i.e., in time largely independent of the entire scene model's complexity. Occlusion culling is one of the key techniques for output-sensitive rendering. We generalize existing occlusion culling algorithms, intended for static scenes, to handle dynamic scenes having numerous moving objects. The data structure used by an occlusion culling method is updated to reflect the objects' possible positions. To avoid updating the structure for every dynamic object at each frame, a temporal bounding volume (TBV) is created for each occluded dynamic object, using some known constraints on the object's motion. The TBV is inserted into the structure instead of the object. Subsequently, the object is ignored as long as the TBV is occluded and guaranteed to contain the object. The generalized algorithms' rendering time is linearly affected only by the scene's visible parts, not by hidden parts or by occluded dynamic objects. Our techniques also save communications in distributed graphic systems, e.g., multiuser virtual environments, by eliminating update messages for hidden dynamic objects. We demonstrate the adaptation of two occlusion culling algorithms to dynamic scenes: hierarchical Z-buffering and BSP tree projection     

Robust foreground object segmentation from handheld camera videos with occlusion map

Extracting foreground objects from videos captured by a handheld camera has emerged as a new challenge. While existing approaches aim to exploit several clues such as depth and motion to extract the foreground layer, there are limitations in handling partial movement and cast shadow. In this paper, we bring a novel perspective to address these two issues by utilizing occlusion map introduced by object and camera motion and taking the advantage of interactive image segmentation methods. For partial movement, we treat each video frame as an image and synthesize “seeding” user interactions (i.e., user manually marking foreground and background) with both forward and backward occlusion maps to leverage the advances in high quality interactive image segmentation. For cast shadow, we utilize a paired region based shadow detection method to further refine initial segmentation results by removing detected shadow regions. Experimental results from both qualitative evaluation and quantitative evaluation on the Hopkins dataset demonstrate both the effectiveness and the efficiency of our proposed approach.     

三维成像声纳图像后处理与可视化

针对基于线阵合成孔径技术的三维成像声纳缺乏有效后处理与可视化方法的问题,提出一个适用于三维成像声纳图像后处理与可视化方案。鉴于水体和地层2种介质具有不同的声学特性,使用组合算法分离水体和地层数据,利用基于边缘梯度均值约束的三维自适应区域生长算法分别对水底和地层目标进行检测,融合水体和地层数据,采用光线投射法进行三维实验数据的可视化。湖试和近海海试结果表明,该方案能够适应声成像中水体与地层的不同特点,有效检测目标,提高声纳图像的表现能力。     

An interactive handheld spherical 3D object display system

Traditional display systems usually display 3D objects on static screens (monitor, wall, etc.) and the manipulation of virtual objects by the viewer is usually achieved via indirect tools such as keyboard or mouse. It would be more natural and direct if we display the object onto a handheld surface and manipulate it with our hands as if we were holding the real 3D object. In this paper, we propose a prototype system by projecting the object onto a handheld foam sphere. The aim is to develop an interactive 3D object manipulation and exhibition tool without the viewer having to wear spectacles. In our system, the viewer holds the sphere with his hands and moves it freely. Meanwhile we project well-tailored images onto the sphere to follow its motion, giving the viewer a virtual perception as if the object were sitting inside the sphere and being moved by the viewer. The design goal is to develop a low-cost, real-time, and interactive 3D display tool. An off-the-shelf projector-camera pair is first calibrated via a simple but efficient algorithm. Vision-based methods are proposed to detect the sphere and track its subsequent motion. The projection image is generated based on the projective geometry among the projector, sphere, camera and the viewer. We describe how to allocate the view spot and warp the projection image. We also present the result and the performance evaluation of the system.     

一种鲁棒的多特征融合目标跟踪新算法

仅利用单一的目标特征进行跟踪是大多数跟踪算法鲁棒性不高的重要原因。提出了一种新的多特征融合目标跟踪算法,该算法将目标的颜色、纹理、边缘、运动特征统一使用直方图模型进行描述,以降低算法受目标形变和部分遮挡的影响,在Auxiliary粒子滤波框架内将所有特征观测进行概率融合,以突出状态后验分布中目标真实状态对应的峰值,从而有效避免了复杂背景的干扰,并给出了一种有效的融合系数计算方法,使融合结果更加准确可靠。实验结果表明,该算法能同时处理刚性与非刚性目标的跟踪,较单一特征的跟踪算法具有明显的优势,对复杂背景下的跟踪具有较高的鲁棒性。与现有多特征融合算法的比较也证明了本文算法的有效性。     

CrossbowCam: a handheld adjustable multi-camera system

Multimedia Tools and Applications - This paper presents a novel multi-functional, low-cost handheld multi-camera system (one dimensional camera array) - “CrossbowCam”. The CrossbowCam...     

Wide-baseline view synthesis for light-field display based on plane-depth-fused sweep volume

The advancements in three-dimensional (3D) display technology have led to a wide interest in light-field display. However, the need to simultaneously capture a large number of object views made content generation for light-field displays still a bottleneck. In this paper, we propose a method for light-field content generation based on plane-depth-fused sweep volume (PDFSV), focusing on handling wide-baseline views and exhibiting scene generalization when the camera array remains unchanged. Specifically, the proposed PDFSV exploits the prior depth of the images captured by a 4 × 4 spherical camera array to represent 3D information of scenes. Then two optimized sequential convolutional neural networks (CNN) are employed for implicit depth modeling and final color calculation, respectively. By doing these, the prior depth facilitates the synthesis of regions with complex textures in the target view. We produce a Wide-baseline Multi-view Image Set (WMIS) which has a field of view (FOV) angle reaching 54°and could be publicly available for access. In our experiments, we use only the 4 vertex views as input. Results demonstrate that the proposed approach can synthesize high-quality views at arbitrary positions between sparse views, outperforming existing neural-radiance-fields-based (NeRF-based) methods. Finally, we conduct autostereoscopic display experiments, achieving satisfactory results.     

基于目标远程探测的合成孔径处理算法研究

给出了基于目标远程探测技术的BDSAP合成孔径处理算法,并且在消声水池中进行了8阵元合成48阵元以及单阵元合成8阵元的逆合成孔径实验研究。结果表明合成孔径处理与常规物理孔径处理相比具有较好的鲁棒性,并且在空间相关长度大于合成孔径长度以及信号时域相关长度大于合成孔径所需时间的水下或海洋环境里,合成阵增益与等长的物理阵增益基本相等。     

Volumetric ambient occlusion for volumetric models

This paper presents new algorithms to compute ambient occlusion for volumetric data. Ambient occlusion is used in video-games and film animation to mimic the indirect lighting of global illumination. We extend a novel interpretation of ambient occlusion to volumetric models that measures how big portion of the tangent sphere of a surface belongs to the set of occluded points, and propose statistically robust estimates for the ambient occlusion value. The data needed by this estimate can be obtained by separable filtering of the voxel array. As ambient occlusion is meant to obtain global illumination effects, it can provide decisive clues in interpreting the data. The new algorithms obtain smooth shading and can be computed at interactive rates, being thus appropriate for dynamic models exploration.     

移动机器人多目标彩色视觉跟踪系统

机器人视觉系统利用颜色、形状等信息来识别环境目标，但是难点在于识别的 鲁棒性和实时性的保证．本文采用全自主移动机器人为平台，提出了一种硬件成本低廉的、 基于颜色学习的实时多目标视觉跟踪系统，并提出了一种新颖的目标颜色学习和跟踪算法． 该视觉系统已成功集成在自主移动机器人系统中，非结构环境下的动态目标跟踪实验表明了 系统的实时性和鲁棒性．     

TUOD 遮挡图像库的设计与实现

遮挡问题是复杂场景图像中一个普遍存在的现象,探索遮挡对图像认知的影响规 律、建立具有抗遮挡能力的认知模型直接关系到计算机视觉技术的实际应用,是一个迫切需要 解决的科学问题。通过研究复杂场景图像中的遮挡问题,探索遮挡对图像认知的影响规律,建 立一个评估检测识别算法的抗遮挡能力、研究图像认知模型及抗遮挡规律的 TUOD (Tsinghua University Occlusion Database)遮挡图像库。首先,根据遮挡对图像识别的影响,提出遮挡部件、 遮挡面积、遮挡关系、遮挡复杂度 4 个维度的图像遮挡属性,建立了图像遮挡程度量化标准; 其次,基于遮挡维度提出一个新的层次化图像库组织结构,以此为基础进行数据库构建。从 PASCAL VOC 和 ImageNet 中进行图像筛选和处理,构建了一个包括飞机、车辆、人、动物 4 大类,共 2 100 张图片的 TUOD 遮挡图像库。利用 TUOD 图像库,结合机器学习理论,通过实 验比较分析不同遮挡维度对 Faster R-CNN 算法的影响。实验表明,TUOD 遮挡图像库能够为算 法的抗遮挡能力提供量化评估标准。TUOD 遮挡图像库的建立为提高抗遮挡算法的性能奠定了 基础,具有实用性。     

基于最小冗余线阵的阵列扩展方法

针对均匀传感器线阵应用被动合成孔径技术进行扩展阵列中存在阵元数较多利用率较低,且噪声累积误差问题,提出了一种基于最小冗余线阵的阵列扩展方法.该算法将四阶累积量方法和被动合成孔径技术相结合,能够采用较少的阵元获得较大的阵列有效孔径.先用四阶累积量方法对最小冗余阵作匀速直线运动采集的连续测量量作预处理,获得虚拟均匀线阵效果...