3D object segmentation using B-Surface

3D object segmentation is important in computer vision such as target detection in biomedical image analysis. A new method, called B-Surface algorithm, is generated for 3D object segmentation. An improved 3D external force field combined with the normalized GVF is utilized. After the initialization of a surface model near the target, B-Surface starts to deform to locate the boundary of the object. First, it overcomes the difficulty that comes from analyzing 3D volume image slice by slice. And the speed of B-Surface deformation is enhanced since the internal forces are not needed to compute in every iteration deformation step. Next, the normal at every surface point can be calculated easily since B-Surface is a continuous deformable model. And it has the ability to achieve high compression ratio (ratio of data to parameters) by presenting the whole surface with only a relatively small number of control points. Experimental results and analysis are presented in this paper. We can see that the B-Surface algorithm can find the surface of the target efficiently.     

Partitioning 3D surface meshes using watershed segmentation

This paper describes a method for partitioning 3D surface meshes into useful segments. The proposed method generalizes morphological watersheds, an image segmentation technique, to 3D surfaces. This surface segmentation uses the total curvature of the surface as an indication of region boundaries. The surface is segmented into patches, where each patch has a relatively consistent curvature throughout, and is bounded by areas of higher, or drastically different, curvature. This algorithm has applications for a variety of important problems in visualization and geometrical modeling including 3D feature extraction, mesh reduction, texture mapping 3D surfaces, and computer aided design     

A reinforcement agent for object segmentation in ultrasound images

The principal contribution of this work is to design a general framework for an intelligent system to extract one object of interest from ultrasound images. This system is based on reinforcement learning. The input image is divided into several sub-images, and the proposed system finds the appropriate local values for each of them so that it can extract the object of interest. The agent uses some images and their ground-truth (manually segmented) version to learn from. A reward function is employed to measure the similarities between the output and the manually segmented images, and to provide feedback to the agent. The information obtained can be used as valuable knowledge stored in the Q-matrix. The agent can then use this knowledge for new input images. The experimental results for prostate segmentation in trans-rectal ultrasound images show high potential of this approach in the field of ultrasound image segmentation.     

Geometric techniques for 3D tracking of ultrasound sensor,tumor segmentation in ultrasound images,and 3D reconstruction

This paper presents different methods, some based on geometric algebra, for ultrasound probe tracking in endoscopic images, 3D allocation of the ultrasound probe, ultrasound image segmentation (to extract objects like tumors), and 3D reconstruction of the surface defined by a set of points. The tracking of the ultrasound probe in endoscopic images is done with a particle filter and an auxiliary method based on thresholding in the HSV space. The 3D pose of the ultrasound probe is calculated using conformal geometric algebra (to locate each slide in 3D space). Each slide (ultrasound image) is segmented using two methods: the level-set method and the morphological operators approach in order to obtain the object we are interested in. The points on the object of interest are obtained from the segmented ultrasound images, and then a 3D object is obtained by refining the convex hull. To do that, a peeling process with an adaptive radius is applied, all of this in the geometric algebra framework. Results for points from ultrasound images, as well as for points from objects from the AimatShape Project, are presented (A.I.M.A.T.S.H.A.P.E. – Advanced an Innovative Models And Tools for the development of Semantic-based systems for Handling, Acquiring, and Processing knowledge Embedded in multidimensional digital objects).     

Semantic part segmentation method based 3D object pose estimation with RGB-D images for bin-picking

3D object pose estimation for grasping and manipulation is a crucial task in robotic and industrial applications. Robustness and efficiency for robotic manipulation are desirable properties that are still very challenging in complex and cluttered scenes, because 3D objects have different appearances, illumination and occlusion when seen from different viewpoints. This article proposes a Semantic Point Pair Feature (PPF) method for 3D object pose estimation, which combines the semantic image segmentation using deep learning with the voting-based 3D object pose estimation. The Part Mask RCNN ispresented to obtain the semantic object-part segmentation related to the point cloud of object, which is combined with the PPF method for 3D object pose estimation. In order to reduce the cost of collecting datasets in cluttered scenes, a physically-simulated environment is constructed to generate labeled synthetic semantic datasets. Finally, two robotic bin-picking experiments are demonstrated and the Part Mask RCNN for scene segmentation is evaluated through the constructed 3D object datasets. The experimental results show that the proposed Semantic PPF methodimproves the robustness and efficiency of 3D object pose estimation in cluttered scenes with partial occlusions.     

3D video components generation using object tracking technique

2D-to-3D conversion that would be a solution of the lack of 3D contents has been a worthy and challenging research field. In this paper, we propose a computer interactive conversion method to capture components which is used to generate 3D sequences. First, we divide the key frame into foreground and background, and then label the objects by convenient computer interactive operation. Depth information of objects is labeled after segmentation. Second, we use object tracking technique which synthesizes the advantages of kernel-based mean shift tracker and contour tracker to accomplish object depth capture for non-key frame. Finally, all the 3D information is prepared to render 3D sequences. After all, we propose our future work direction: a 2D-to-3D system which can generate 3D sequence interactively.     

Automatic 3D object segmentation in multiple views using volumetric graph-cuts

We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object’s colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence.     

3维牙颌模型牙齿分割的路径规划方法

目的 从3维牙颌模型上分割出单颗牙齿是计算机辅助正畸系统的重要步骤。由于3维测量分辨率和网格重建精度的有限性,三角网格牙颌模型上牙龈和牙缝边界往往融合在一起,使得单颗牙齿的自动分割变得极为困难。传统方法容易导致分割线断裂、分支干扰等问题,且手工交互较多,为此提出一种新颖的基于路径规划技术的单颗牙齿自动分割方法。方法 为避免在探测边界时牙龈和牙缝相互干扰,采用牙龈路径和牙缝路径分开规划策略。首先基于离散曲率分析和一种双重路径规划法搜索牙龈分割路径,并基于搜索到的牙龈路径利用图像形态学和B样条拟合技术构建牙弓曲线;然后综合牙龈路径和牙弓曲线的形态特征探测牙龈路径上的牙缝凹点以划界每颗牙齿的牙龈边界轮廓,并通过匹配和搜索牙龈边界轮廓上颊舌侧凹点间的最优路径确定齿间牙缝边界路径;最后细化整个路径以获取每颗牙齿精确的封闭分割轮廓。结果 对不同畸形程度的患者牙颌模型进行分割实验,结果表明,本文方法对于严重畸形的牙齿能够产生正确的分割结果,而且简单快速,整个分割过程基本能够控制在20 s以内。和现有方法相比,本文方法具有较少的人工干预和参数调整,除了在个别牙齿边界较为模糊的位置需要手动调整外,大部分情况都是自动的。结论 提出的路径规划方法具有强大的抗干扰能力,能够有效克服牙缝牙沟等分支干扰以及分割线断裂等问题,最大程度地减少人工干预,适用于各类畸形牙患者模型的牙齿分割。     

基于语义分割引导的三维目标检测

三维目标检测是计算机视觉领域的热门研究内容之一。在自动驾驶系统中,三维目标检测技术通过捕获周围的点云信息与RGB图像信息,对周围物体进行检测,从而为车辆规划下一步的行进路线。因此,通过三维目标检测实现对周边环境的精准检测与感知是十分重要的。针对三维目标检测技术中随机采样算法导致前景点丢失的问题,首先提出了基于语义分割的随机采样算法,通过预测的语义特征指导采样过程,提升了前景点的采样比重,进而提高了三维目标检测精度;其次,针对三维目标检测定位置信度与分类置信度不一致的问题,提出了CL联合损失,使得网络倾向于选择定位置信度与分类置信度都高的3D候选框,避免了传统的NMS仅考虑分类置信度所带来的歧义问题。在KITTI三维目标检测数据集进行了实验,结果表明,该方法能够在简单、中等、困难3个难度下均获得精度的提升,从而验证了其在三维目标检测任务中的有效性。     

立体视频对象分割及其三维重建算法研究*

为更加有效分析立体视频对象,本文提出了一种基于离散冗余小波变换的立体视频对象分割算法,首先采用离散冗余小波变换提取特征点结合DT网格技术的视差估计方法,获得了可靠的视差场,再利用视差信息对立体视频中静止对象进行分割。对于立体视频序列中的运动对象,采用离散冗余小波提取运动区域的方法进行分割。实验结果表明,本算法对有重叠的多视频对象具有较好的分割效果,可同时分割静止物体和运动物体,具有较好的精确性和鲁棒性。对于分割出的立体视频对象,结合深度信息对其进行三维重建,得到较好的三维效果。     

3-D object segmentation using ant colonies

3-D object segmentation is an important and challenging topic in computer vision that could be tackled with artificial life models.A Channeler Ant Model (CAM), based on the natural ant capabilities of dealing with 3-D environments through self-organization and emergent behaviours, is proposed.Ant colonies, defined in terms of moving, pheromone laying, reproduction, death and deviating behaviours rules, is able to segment artificially generated objects of different shape, intensity, background.The model depends on few parameters and provides an elegant solution for the segmentation of 3-D structures in noisy environments with unknown range of image intensities: even when there is a partial overlap between the intensity and noise range, it provides a complete segmentation with negligible contamination (i.e., fraction of segmented voxels that do not belong to the object). The CAM is already in use for the automated detection of nodules in lung Computed Tomographies.     

Tissue segmentation in ultrasound images by using genetic algorithms

This paper presents a genetic based incremental neural network (GINeN) for the segmentation of tissues in ultrasound images. Performances of the GINeN and the Kohonen network are investigated for tissue segmentation in ultrasound images. Feature extraction is carried out by using continuous wavelet transform. Pixel intensities at the same spatial location on 12 wavelet planes and on the original image are considered as features, leading to 13-dimensional feature vectors. The same training set is used for the training of the Kohonen network and the GINeN.

This paper proposes the use of wavelet transform and genetic based incremental neural network together in order to increase the segmentation performance. It is observed that genetic based incremental neural network gives satisfactory segmentation performance for ultrasound images.     

Contour segmentation in 2D ultrasound medical images with particle filtering

Object segmentation in medical images is an actively investigated research area. Segmentation techniques are a valuable tool in medical diagnostics for cancer tumours and cysts, for planning surgery operations and other medical treatment. In this paper, a Monte Carlo algorithm for extracting lesion contours in ultrasound medical images is proposed. An efficient multiple model particle filter for progressive contour growing (tracking) from a starting point is developed, accounting for convex, non-circular forms of delineated contour areas. The driving idea of the proposed particle filter consists in the incorporation of different image intensity inside and outside the contour into the filter likelihood function. The filter employs image intensity gradients as measurements and requires information about four manually selected points: a seed point, a starting point, arbitrarily selected on the contour, and two additional points, bounding the measurement formation area around the contour. The filter performance is studied by segmenting contours from a number of real and simulated ultrasound medical images. Accurate contour segmentation is achieved with the proposed approach in ultrasound images with a high level of speckle noise.     

Fast segmentation of ultrasound images using robust Rayleigh distribution decomposition

The segmentation of left ventricle in ultrasound imaging of human heart would provide an important clinical parameter for the evaluation of cardiac functions including volume stroke or ejection fraction and wall motion tracking. We propose a fast segmentation method to reduce laborious manual efforts and conveniently provide robust and stable cardiac quantification to users. The proposed method provides a very simple energy functional form using a predetermined Rayleigh distribution parameter so that the corresponding steepest descent approach with some shape constraints on contour is still capable of fast segmentation. We present several experimental results on two-dimensional echocardiography data for the performance of the proposed model. The experiments show that the proposed model is especially useful when a part of target boundary is seriously corrupted.     

深层聚合残差密集网络的超声图像左心室分割

目的 超声图像是临床医学中应用最广泛的医学图像之一,但左心室超声图像一般具有强噪声、弱边缘和组织结构复杂等问题,其图像分割难度较大。临床上需要一种效率高、质量好的超声图像左心室分割算法。本文提出一种基于深层聚合残差密集网络（deep layer aggregation for residual dense network,DLA-RDNet）的超声图像左心室分割算法。方法 对获取的超声图像进行形态学操作,定位目标区域,得到目标图像。构建残差密集网络（residual dense network,RDNet）用于提取图像特征,并将RDNet得到的层次信息通过深层聚合（deep layer aggregation,DLA）的方式紧密融合到一起,得到分割网络DLA-RDNet,用于实现对超声图像左心室的精确分割。通过深监督（deep supervision,DS）方式为网络剪枝,简化网络结构,提升网络运行速度。结果 数据测试集的实验结果表明,所提算法平均准确率为95.68%,平均交并比为97.13%,平均相似性系数为97.15%,平均垂直距离为0.31 mm,分割轮廓合格率为99.32%。与6种分割算法相比,所提算法的分割精度更高。在测试阶段,每幅图像仅需不到1 s的时间即可完成分割,远远超出了专业医生的分割速度。结论 提出了一种深层聚合残差密集神经网络对超声图像左心室进行分割,通过主、客观对比实验表明本文算法的有效性,能够较对比方法更实时准确地对超声图像左心室进行分割,符合临床医学中超声图像左心室分割的需求。     

Deep residual neural network based PointNet for 3D object part segmentation

Multimedia Tools and Applications - Point cloud segmentation is the premise and basis of many 3D perception tasks, such as intelligent driving, object detection and recognition, scene recognition...     

Graph-cut based interactive segmentation of 3D materials-science images

Segmenting materials’ images is a laborious and time-consuming process, and automatic image segmentation algorithms usually contain imperfections and errors. Interactive segmentation is a growing topic in the areas of image processing and computer vision, which seeks to find a balance between fully automatic methods and fully-manual segmentation processes. By allowing minimal and simplistic interaction from the user in an otherwise automatic algorithm, interactive segmentation is able to simultaneously reduce the time taken to segment an image while achieving better segmentation results. Given the specialized structure of materials’ images and level of segmentation quality required, we show an interactive segmentation framework for materials’ images that has three key contributions: (1) a multi-labeling approach that can handle a large number of structures while still quickly and conveniently allowing manual addition and removal of segments in real-time, (2) multiple extensions to the interactive tools which increase the simplicity of the interaction, and (3) a web interface for using the interactive tools in a client/server architecture. We show a full formulation of each of these contributions and example results from their application.