立体视频对象分割及其三维重建算法研究*

为更加有效分析立体视频对象,本文提出了一种基于离散冗余小波变换的立体视频对象分割算法,首先采用离散冗余小波变换提取特征点结合DT网格技术的视差估计方法,获得了可靠的视差场,再利用视差信息对立体视频中静止对象进行分割。对于立体视频序列中的运动对象,采用离散冗余小波提取运动区域的方法进行分割。实验结果表明,本算法对有重叠的多视频对象具有较好的分割效果,可同时分割静止物体和运动物体,具有较好的精确性和鲁棒性。对于分割出的立体视频对象,结合深度信息对其进行三维重建,得到较好的三维效果。     

An edge-based approach for the segmentation of 3-D range images of small industrial-like objects

An edge-based approach for the segmentation of 3-D range images of small industrial-like objects is presented. It is illustrated that the gradient edge operators are mostly suitable for detecting jump boundaries. They may produce misleading responses and may also miss detecting roof edges. We also show, using numerical analysis techniques, that the residual norms of fitting planes at surface points where the residuals are measured in the z-direction have large magnitudes at the jump boundaries. At the folds or creases they have large enough magnitudes but are shifted by one pixel from the crease. We show however that the residual norms of the fitting planes where the residuals are measured in the direction normal to the planes are more suitable for detecting the various image edges. Experimental results for the segmentation of 3-D range images of small industrial-like objects, using the histograms of such residual norms are given with comments.     

具有强鲁棒性的三维对象多视变分分割方法

针对校准摄像机采集系列图像的三维分割重构问题,提出了一种新的面向概率描述的变分方法。首先,计算系列图像的极大似然曲面,可重构与分割保持一致的三维曲面;接着,融合联合概率,可重构目标对象及图像背景的平均强度及标准差;最后,采用水平集框架,可实现对曲面能量方程的数值模拟。该方法适用于复杂拓扑结构重构及噪声数据处理。实验结果表明,该方法实用性好,鲁棒性强,对任意三维对象的分割重构效果较形状雕刻方法及体视分割方法理想。     

A multi-plane approach for text segmentation of complex document images

This study presents a new method, namely the multi-plane segmentation approach, for segmenting and extracting textual objects from various real-life complex document images. The proposed multi-plane segmentation approach first decomposes the document image into distinct object planes to extract and separate homogeneous objects including textual regions of interest, non-text objects such as graphics and pictures, and background textures. This process consists of two stages—localized histogram multilevel thresholding and multi-plane region matching and assembling. Then a text extraction procedure is applied on the resultant planes to detect and extract textual objects with different characteristics in the respective planes. The proposed approach processes document images regionally and adaptively according to their respective local features. Hence detailed characteristics of the extracted textual objects, particularly small characters with thin strokes, as well as gradational illuminations of characters, can be well-preserved. Moreover, this way also allows background objects with uneven, gradational, and sharp variations in contrast, illumination, and texture to be handled easily and well. Experimental results on real-life complex document images demonstrate that the proposed approach is effective in extracting textual objects with various illuminations, sizes, and font styles from various types of complex document images.     

Reconstruction of 3D Object Meshes from Silhouette Images

In this work we propose a method for computing mesh representations of 3D objects reconstructed from a set of silhouette images. Our method is based on the polygonization of volumetric reconstructions by using a modified version of the dual contouring method. In order to apply dual contouring on volumetric reconstruction from silhouettes we devised a method that is able to determine the discrete topology of the surface in relation to the octree cells. We also developed a new scheme for computing hermitian data representing the intersections of conic volumes with the octree cells and their corresponding normals with subpixel accuracy. Due to the discrete and extremely noisy nature of the data used in the reconstruction we had to devise a different criterion for mesh simplification that applies topological consistency tests only when the geometric error measure is beyond a given tolerance. We present results of the application of the proposed method in the extraction of a mesh corresponding to the surface of objects of a real scene.     

自动分割视频序列中运动物体的新算法：多特征联合方法

视频编码标准MPEG-4增加了适于多种应用的基于视频内容的功能,为了支持这一功能和提高编码效率,MPEG-4将视频序列中的每一帧分解成视频对象面(VOP);另外,由于基于内容的视频检索和视频监控系统均期望用分割出的关键视频对象紧致地表示一个序列,同时由于视频分割技术在模式识别、计算机视觉等领域也得到了广泛的应用,因此,分割视频运动物体并跟踪运动物体的变化变得至关重要.为了对视频中运动物体进行有效的分割,在帧差图象的基础上,采用Canny边缘检测和随机信号的高阶矩检测相结合的方法,来自动分割视频序列的前景区域和背景区域,并在前景区域中应用区域生长法进行颜色分割,以精确提取运动物体的边缘;还利用边缘和颜色特征来对分割出的运动物体建立模板,用于解决非刚体运动中局部暂时停止运动的情况.实验结果表明,此方法可以有效地分割运动物体,并能跟踪运动物体的变化.     

二维遗传算法用于图象动态分割

为了有效地对受噪声影响的图象进行分析,提出了两种基于二维遗传算法的图象动 态分割算法.在这些算法中:1)分别采用了以阈值曲面和模糊隶属度曲面为染色体的二维染 色体编码方式;2)采用了全局阈值化算法和模糊集合理论初始化种群;3)采用Hopfield网络 的能量函数形式,结合FCM算法和现有阈值化算法中的一般性分割准则构造适应度函数. 利用实际图象将所提出的算法与一些典型算法进行了分割比较实验,结果表明所提算法有较 好的抗噪效果.     

Converting discrete images to partitioning trees

The discrete space representation of most scientific datasets, generated through instruments or by sampling continuously defined fields, while being simple, is also verbose and structureless. We propose the use of a particular spatial structure, the binary space partitioning tree as a new representation to perform efficient geometric computation in discretely defined domains. The ease of performing affine transformations, set operations between objects, and correct implementation of transparency makes the partitioning tree a good candidate for probing and analyzing medical reconstructions, in such applications as surgery planning and prostheses design. The multiresolution characteristics of the representation can be exploited to perform such operations at interactive rates by smooth variation of the amount of geometry. Application to ultrasound data segmentation and visualization is proposed. The paper describes methods for constructing partitioning trees from a discrete image/volume data set. Discrete space operators developed for edge detection are used to locate discontinuities in the image from which lines/planes containing the discontinuities are fitted by using either the Hough transform or a hyperplane sort. A multiresolution representation can be generated by ordering the choice of hyperplanes by the magnitude of the discontinuities. Various approximations can be obtained by pruning the tree according to an error metric. The segmentation of the image into edgeless regions can yield significant data compression. A hierarchical encoding schema for both lossless and lossy encodings is described     

Continuous global optimization in surface reconstruction from an oriented point cloud

We introduce a continuous global optimization method to the field of surface reconstruction from discrete noisy cloud of points with weak information on orientation. The proposed method uses an energy functional combining flux-based data-fit measures and a regularization term. A continuous convex relaxation scheme assures the global minima of the geometric surface functional. The reconstructed surface is implicitly represented by the binary segmentation of vertices of a 3D uniform grid and a triangulated surface can be obtained by extracting an appropriate isosurface. Unlike the discrete graph-cut solution, the continuous global optimization entails advantages like memory requirements, reduction of metrication errors for geometric quantities, and allowing globally optimal surface reconstruction at higher grid resolutions. We demonstrate the performance of the proposed method on several oriented point clouds captured by laser scanners. Experimental results confirm that our approach is robust to noise, large holes and non-uniform sampling density under the condition of very coarse orientation information.     

Robust feature detection and local classification for surfaces based on moment analysis

The stable local classification of discrete surfaces with respect to features such as edges and corners or concave and convex regions, respectively, is as quite difficult as well as indispensable for many surface processing applications. Usually, the feature detection is done via a local curvature analysis. If concerned with large triangular and irregular grids, e.g., generated via a marching cube algorithm, the detectors are tedious to treat and a robust classification is hard to achieve. Here, a local classification method on surfaces is presented which avoids the evaluation of discretized curvature quantities. Moreover, it provides an indicator for smoothness of a given discrete surface and comes together with a built-in multiscale. The proposed classification tool is based on local zero and first moments on the discrete surface. The corresponding integral quantities are stable to compute and they give less noisy results compared to discrete curvature quantities. The stencil width for the integration of the moments turns out to be the scale parameter. Prospective surface processing applications are the segmentation on surfaces, surface comparison, and matching and surface modeling. Here, a method for feature preserving fairing of surfaces is discussed to underline the applicability of the presented approach.     

基于直方图预处理与BF算法的含噪图像分割

图像中的噪声会直接影响图像分割质量,为快速、准确地识别含噪图像中的目标,提出一种基于直方图预处理与BF算法的含噪图像分割方法。该方法通过小波变换抑制图像中的噪声,分析增强图像的直方图特点以缩小分割阈值的分布范围,以二维最大类间方差为原则设计分割目标函数,利用BF算法快速搜索最优分割阈值。实验结果表明,该方法在收敛速度、稳定性和分割效果三个方面均优于基于遗传算法、人工鱼群算法等其他群体智能的分割方法。     

基于视频对象的视频内容分级描述模型

本文提出了一种基于视频对象的视频内容分级描述模型.视频序列首先被分成一个个的镜头,在每个镜头内对视频对象进行分割和跟踪.按照镜头、视频对象、视频对象平面和视频对象区域四级框架提取特征,对视频内容进行分级描述.本文对视频内容信息的描述可以用于视频检索、视频注释等应用.     

Geometric segmentation of 3D scanned surfaces

The geometric segmentation of a discrete geometric model obtained by the scanning of real objects is affected by various problems that make the segmentation difficult to perform without uncertainties. Certain factors, such as point location noise (coming from the acquisition process) and the coarse representation of continuous surfaces due to triangular approximations, introduce ambiguity into the recognition process of the geometric shape. To overcome these problems, a new method for geometric point identification and surface segmentation is proposed.The point classification is based on a fuzzy parameterization using three shape indexes: the smoothness indicator, shape index and flatness index. A total of 11 fuzzy domain intervals have been identified and comprise sharp edges, defective zones and 10 different types of regular points. For each point of the discrete surface, the related membership functions are dynamically evaluated to be adapted to consider, point by point, those properties of the geometric model that affects uncertainty in point type attribution.The methodology has been verified in many test cases designed to represent critical conditions for any method in geometric recognition and has been compared with one of the most robust methods described in the related literature.     

Surface Reconstruction from Unorganized Point Data with Quadrics

We present a reverse engineering method for constructing a surface approximation scheme whose input is a set of unorganized noisy points in space and whose output is a set of quadric patches. The local surface properties, necessary for the subsequent segmentation, are estimated directly from the data using a simple and efficient data structure—the neighborhood graph. Our segmentation scheme, based on principal curvatures, constructs initial point subsets, which may be enlarged or further subdivided based on associated approximation error estimates obtained through approximation of the initial segments by quadric surfaces. Our method is highly efficient and produces a high‐quality piecewise quadric surface approximation of engineering objects, which we demonstrate for several simple and complex example data sets.     

交互式分割视频运动对象的研究与实现

在 MPEG- 4视频编码标准中 ,为了实现基于视频内容的交互功能 ,视频序列的每一帧由视频对象面来表示 ,而生成视频对象面 ,需要对视频序列中运动对象进行有效分割 ,并跟踪运动对象随时间的变化 .在视频分割方法中 ,交互式分割视频对象能满足分割的效率与质量指标要求 ,因此提出了一种交互分割与自动跟踪相结合的方式来分割视频语义对象 ,即在初始分割时 ,依据用户的交互与形态学的分水线分割算法相结合提取视频对象轮廓 ,并用改进的轮廓跟踪方法有效提高视频对象轮廓的精度 ;对后续帧的跟踪 ,采用六参数仿射变换跟踪运动对象轮廓的变化 ,用平移估算的运动矢量作为初始值 ,计算六参数仿射变换的参数 .实验结果表明 ,该方法能有效地分割并跟踪视频运动对象     

Estimation of planes of a rock mass in a gallery wall from point cloud data based on MD PSO

LiDAR (laser imaging detection and ranging) has been developed to measure the distance of the mesh of points on an object with a high level of accuracy. It provides high-resolution point cloud data as a result of distance measurement. Detailed 3D shapes of objects can be estimated from point cloud data. LiDAR has been used to identify discontinuities in a rock mass of a tunnel gallery wall. To identify discontinuities, it is necessary to approximate the rock mass surface with small planes. Normal vectors of the planes are important to identify discontinuities. We developed an algorithm for estimation of planes based on multi-dimensional particle swarm optimization (MD PSO) from point cloud data. Point cloud data were segmented into bounding boxes and grouped into clusters by MD PSO. Planes were estimated using the least squares method for point cloud data in the respective clusters. The newly developed algorithm based on MD PSO was evaluated using point cloud data obtained from a gallery wall. Evaluation was carried out in comparison with the previous developed variable-box segmentation (VBS) algorithm. The MD PSO-based algorithm showed a 7% higher accuracy than that of the VBS algorithm.     

一种提花织物图像的有限元分割算法

提花织物图像分割是提花图案设计的关键,曲线演化模型是一种流行的图像分割方法,但是该方法无法检测含噪环境下的图像特征.由于Mumford-Shah(MS)模型能够在噪声环境下对不连续边集进行检测,因此它比曲线演化模型更适于对含噪提花织物图像的分割.提出一种结合有限元法和拟牛顿法的MS模型数值求解算法,并有效用于含噪提花织物图像的分割.首先定义了自适应三角剖分空间上的离散MS模型,并在每次迭代前对有限元网格进行自适应调整,以提高迭代的性能.接着采用拟牛顿最小化方法,通过收敛意义上的离散有限元逼近得到离散MS模型的最小值.该算法被用到含噪提花织物图像的分割中,取得了良好的效果.     

Multiresolution video object extraction fitted to scalable wavelet-based object coding

To enable content based functionalities in video processing algorithms, decomposition of scenes into semantic objects is necessary. A semi-automatic Markov random field based multiresolution algorithm is presented for video object extraction in a complex scene. In the first frame, spatial segmentation and user intervention determine objects of interest. The specified objects are subsequently tracked in successive frames and newly appeared objects/regions are also detected. The video object extraction algorithm includes discrete wavelet transform decomposition multiresolution Markov random field (MRF)-based spatial segmentation with emphasis on border smoothness at different resolutions, and an MRF-based backward region classification that determines the tracked objects in the scene. Finally, a motion constraint, embedded in the region classifier, determines the newly appeared objects/regions and completes the proposed algorithm towards an efficient video segmentation algorithm. The results are applicable for generic segmentation applications, however the proposed multiresolution video segmentation algorithm supports scalable object-based wavelet coding in particular. Moreover, compared to traditional object extraction algorithms, it produces smoother and more visually pleasing shape masks at different resolutions. The proposed effective multiresolution video object extraction method allows for larger motion, better noise tolerance and less computational complexity     

改进法线方向的点云实时分割提取平面方法研究

为了提高服务机器人在室内环境中目标物体抓取的效率,提出一种针对三维点云数据改进表面法线估计的方法。重点研究了如何提高使用三维点云数据实时性和准确性,这是机器人抓取过程的关键因素。使用消费级别的传感器kinectV2获取点云数据,提出使用积分图像降低滤波区域内像素的遍历消耗的时间,从而提高实时性;采用动态调整平滑区域的大小以及分配权重的方法平滑处理点云数据的噪声,提高法线估计精度。依据改进后的法线方向,利用区域生长算法对不同方向的法线进行分类,然后对点云数据实施分割提取平面。最后,采用提出的方法和点云库中的相关算法,进行了平面分割提取测试比较。结果表明,利用kinectV2获取的深度数据（1-5米）比目前点云库中的平面分割算法数据精度更高、实时性更强。     

一种快速的视频序列运动分割方法

提出一种视频序列运动分割的实时方法,该方法通过两次背景更新提取运动前景,其中长程背景更新检测噪声运动区域,并将长时间停留在场景中的物体纳入背景范围,短程背景更新利用单高斯分布模型实现对光线缓慢变化的适应。采用基于颜色空间的方法实现对阴影的消除,通过投影分割提取出不同的运动目标。试验证明该方法能够快速、精确地实现视频序列中多个运动目标的分割,消除背景噪声运动、目标阴影以及场景变化的影响。