Fuzzy segmentation for object‐based image classification

This Letter proposes an object‐based image classification procedure which is based on fuzzy image‐regions instead of crisp image‐objects. The approach has three stages: (a) fuzzification in which fuzzy image‐regions are developed, resulting in a set of images whose digital values express the degree of membership of each pixel to target land‐cover classes; (b) feature analysis in which contextual properties of fuzzy image‐regions are quantified; and (c) defuzzification in which fuzzy image‐regions are allocated to target land‐cover classes. The proposed procedure is implemented using automated statistical techniques that require very little user interaction. The results indicate that fuzzy segmentation‐based methods produce acceptable thematic accuracy and could represent a viable alternative to current crisp image segmentation approaches.     

Re‐Compositable Panoramic Selfie with Robust Multi‐Frame Segmentation and Stitching

It is a challenging task for ordinary users to capture selfies with a good scene composition, given the limited freedom to position the camera. Creative hardware (e.g., selfie sticks) and software (e.g., panoramic selfie apps) solutions have been proposed to extend the background coverage of a selife, but to achieve a perfect composition on the spot when the selfie is captured remains to be difficult. In this paper, we propose a system that allows the user to shoot a selfie video by rotating the body first, then produce a final panoramic selfie image with user‐guided scene composition as postprocessing. Our key technical contribution is a fully Automatic, robust multi‐frame segmentation and stitching framework that is tailored towards the special characteristics of selfie images. We analyze the sparse feature points and employ a spatial‐temporal optimization for bilayer feature segmentation, which leads to more reliable background alignment than previous image stitching techniques. The sparse classification is then propagated to all pixels to create dense foreground masks for person‐background composition. Finally, based on a user‐selected foreground position, our system uses content‐preserving warping to produce a panoramic seflie with minimal distortion to the face region. Experimental results show that our approach can reliably generate high quality panoramic selfies, while a simple combination of previous image stitching and segmentation approaches often fails.     

Content‐Based Colour Transfer

This paper presents a novel content‐based method for transferring the colour patterns between images. Unlike previous methods that rely on image colour statistics, our method puts an emphasis on high‐level scene content analysis. We first automatically extract the foreground subject areas and background scene layout from the scene. The semantic correspondences of the regions between source and target images are established. In the second step, the source image is re‐coloured in a novel optimization framework, which incorporates the extracted content information and the spatial distributions of the target colour styles. A new progressive transfer scheme is proposed to integrate the advantages of both global and local transfer algorithms, as well as avoid the over‐segmentation artefact in the result. Experiments show that with a better understanding of the scene contents, our method well preserves the spatial layout, the colour distribution and the visual coherence in the transfer process. As an interesting extension, our method can also be used to re‐colour video clips with spatially‐varied colour effects.     

Motion Panoramas

In this paper we describe a method for analysing video sequences and for representing them as mosaics or panoramas. Previous work on video mosaicking essentially concentrated on static scenes. We generalize these approaches to the case of a rotating camera observing both static and moving objects where the static portions of the scene are not necessarily dominant, as it has been often hypothesized in the past. We start by describing a robust technique for accurately aligning a large number of video frames under unknown camera rotations and camera settings. The alignment technique combines a feature‐based method (initialization and refinement) with rough motion segmentation followed by a colour‐based direct method (final adjustment). This precise frame‐to‐frame alignment allows the dynamic building of a background representation as well as an efficient segmentation of each image such that moving regions of arbitrary shape and size are aligned with the static background. Thus a motion panorama visualizes both dynamic and static scene elements in a geometrically consistent way. Extensive experiments applied to archived videos of track‐and‐field events validate the approach. Copyright © 2004 John Wiley & Sons, Ltd.     

一种新的图像合成方法

提出了一种新的图像分割与合成方法。对于图像的分割,提出了新的切割图法,先是手工把一些明显属于物体和明显属于背景的像素分割出来,然后设置了新的能量函数,求它取最小值时的分割结果。对于图像的合成,提出了梯度比较法,在保留物体基本颜色特征的同时,改变了物体的光亮度,使其与新背景的光亮度一致,从而合成图像的光照效果更真实。     

Hallucinating Stereoscopy from a Single Image

We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre‐segmented image. Rather than attempting full 3D reconstruction or accurate depth map recovery, we hallucinate a rough approximation of the scene's 3D model using a number of simple depth and occlusion cues and shape priors. We begin by depth‐sorting the segments, each of which is assumed to represent a separate object in the scene, resulting in a collection of depth layers. The shapes and textures of the partially occluded segments are then completed using symmetry and convexity priors. Next, each completed segment is converted to a union of generalized cylinders yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state‐of‐the‐art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi‐interactive and automatic single image depth recovery methods.     

Surveillance and human–computer interaction applications of self-growing models

The aim of the work is to build self-growing based architectures to support visual surveillance and human–computer interaction systems. The objectives include: identifying and tracking persons or objects in the scene or the interpretation of user gestures for interaction with services, devices and systems implemented in the digital home. The system must address multiple vision tasks of various levels such as segmentation, representation or characterization, analysis and monitoring of the movement to allow the construction of a robust representation of their environment and interpret the elements of the scene.It is also necessary to integrate the vision module into a global system that operates in a complex environment by receiving images from acquisition devices at video frequency and offering results to higher level systems, monitors and take decisions in real time, and must accomplish a set of requirements such as: time constraints, high availability, robustness, high processing speed and re-configurability.Based on our previous work with neural models to represent objects, in particular the Growing Neural Gas (GNG) model and the study of the topology preservation as a function of the parameters election, it is proposed to extend the capabilities of this self-growing model to track objects and represent their motion in image sequences under temporal restrictions.These neural models have various interesting features such as: their ability to readjust to new input patterns without restarting the learning process, adaptability to represent deformable objects and even objects that are divided in different parts or the intrinsic resolution of the problem of matching features for the sequence analysis and monitoring of the movement. It is proposed to build an architecture based on the GNG that has been called GNG-Seq to represent and analyze the motion in image sequences. Several experiments are presented that demonstrate the validity of the architecture to solve problems of target tracking, motion analysis or human–computer interaction.     

High Reliefs from 3D Scenes

We present a method for synthesizing high reliefs, a sculpting technique that attaches 3D objects onto a 2D surface within a limited depth range. The main challenges are the preservation of distinct scene parts by preserving depth discontinuities, the fine details of the shape, and the overall continuity of the scene. Bas relief depth compression methods such as gradient compression and depth range compression are not applicable for high relief production. Instead, our method is based on differential coordinates to bring scene elements to the relief plane while preserving depth discontinuities and surface details of the scene. We select a user‐defined number of attenuation points within the scene, attenuate these points towards the relief plane and recompute the positions of all scene elements by preserving the differential coordinates. Finally, if the desired depth range is not achieved we apply a range compression. High relief synthesis is semi‐automatic and can be controlled by user‐defined parameters to adjust the depth range, as well as the placement of the scene elements with respect to the relief plane.     

Albedo estimation for scene segmentation

Standard methods of image segmentation do not take into account the three-dimensional nature of the underlying scene. For example, histogram-based segmentation tacitly assumes that the image intensity is piecewise constant, and this is not true when the scene contains curved surfaces. This paper introduces a method of taking 3D information into account in the segmentation process. The image intensities are adjusted to compensate for the effects of estimated surface orientation; the adjusted intensities can be regarded as reflectivity estimates. When histogram-based segmentation is applied to these new values, the image is segmented into parts corresponding to surfaces of contant reflectivity in the scene.     

A novel method for assessing the segmentation quality of high-spatial resolution remote-sensing images

Image segmentation quality significantly affects subsequent image classification accuracy. It is necessary to develop effective methods for assessing image segmentation quality. In this paper, we present a novel method for assessing the segmentation quality of high-spatial resolution remote-sensing images by measuring both area and position discrepancies between the delineated image region (DIR) and the actual image region (AIR) of a scene object. In comparison with the most frequently used area coincidence-based methods, our method can assess the segmentation quality more objectively in that it takes into consideration all image objects intersecting with the AIR of a scene object. Moreover, the proposed method is more convenient to use than the existing boundary coincidence-based methods in that the calculation of the distance between the boundary of the image object and that of the corresponding AIR of the scene object is not required. Another benefit of this method over the two types of method above is that the assessment procedure of the segmentation quality can be conducted with less human intervention. The obtained optimal segmentation result can ensure maximal delineation of the extent of scene objects and can be beneficial to subsequent classification operations. The experimental results have shown the effectiveness of this new method for both segmentation quality assessment and optimal segmentation parameter selection.     

融合注意力和多尺度特征的街景图像语义分割

为了解决在街道场景图像语义分割任务中传统U-Net网络在多尺度类别下目标分割的准确率较低和图像上下文特征的关联性较差等问题,提出一种改进U-Net的语义分割网络AS-UNet,实现对街道场景图像的精确分割.首先,在U-Net网络中融入空间通道挤压激励(spatial and channel squeeze&excitation block, scSE)注意力机制模块,在通道和空间两个维度来引导卷积神经网络关注与分割任务相关的语义类别,以提取更多有效的语义信息;其次,为了获取图像的全局上下文信息,聚合多尺度特征图来进行特征增强,将空洞空间金字塔池化(atrous spatial pyramid pooling, ASPP)多尺度特征融合模块嵌入到U-Net网络中;最后,通过组合使用交叉熵损失函数和Dice损失函数来解决街道场景目标类别不平衡的问题,进一步提升分割的准确性.实验结果表明,在街道场景Cityscapes数据集和Cam Vid数据集上AS-UNet网络模型的平均交并比(mean intersection over union, MIo U)相较于传统U-Net网络分别提...     

基于内容的图象检索中的语义处理方法

基于内容的图象检索系统,其目标是最大限度地减小图象简单视觉特征与用户检索丰富语义之间的“语义鸿沟”,因此图象语义处理则成为基于内容的图象检索进一步发展的关键。为了使人们对基于内容的图象检索中的语义处理方法有个概略了解,首先从图象语义模型和图象语义提取方法这两个方面对利用语义进行图象检索的研究状况进行了总结,并将图象语义模型概括为图象语义知识、图象语义层次模型和语义抽取模型等3个主要组成部分;然后将图象语义提取方法分为用户交互、将查询请求作为语义模板、对象及其空间关系、场景和行为语义及情感语义等类别,同时对其中有代表性的方法进行了详细的分析,还指出了其局限性;最后从对象建模和识别、语义抽取规则和用户检索模型3个方面,阐明了实现图象语义处理所面临的问题,并提出了一些初步的解决思路。     

TouchTone: Interactive Local Image Adjustment Using Point‐and‐Swipe

Recent proliferation of camera phones, photo sharing and social network services has significantly changed how we process our photos. Instead of going through the traditional download‐edit‐share cycle using desktop editors, an increasing number of photos are taken with camera phones and published through cellular networks. The immediacy of the sharing process means that on‐device image editing, if needed, should be quick and intuitive. However, due to the limited computational resources and vastly different user interaction model on small screens, most traditional local selection methods can not be directly adapted to mobile devices. To address this issue, we present TouchTone, a new method for edge‐aware image adjustment using simple finger gestures. Our method enables users to select regions within the image and adjust their corresponding photographic attributes simultaneously through a simple point‐and‐swipe interaction. To enable fast interaction, we develop a memory‐ and computation‐efficient algorithm which samples a collection of 1D paths from the image, computes the adjustment solution along these paths, and interpolates the solutions to entire image through bilateral filtering. Our system is intuitive to use, and can support several local editing tasks, such as brightness, contrast, and color balance adjustments, within a minute on a mobile device.     

A Real-time Scene Segmentation System using Solely Excitatory Oscillator Networks (SEON)

Scene segmentation is one of the most important tasks in research and commercial applications. With the rapid development of Internet, there is an increasing demand for real-time, mobile and autonomous multi-media system to increase the user-friendliness of e-shopping experience and to provide value-added e-services. Traditional scene segmentation techniques, which are mainly sequential and offline, cannot segment images in a real-time manner. This paper introduces a real-time, multi-media system known as Intelligent Java Agent Development Environment (iJADE) Scene Segmentater—an agent-based scene segmentation system using Solely Excitatory Oscillator networks (SEON) for real-time scene segmentation. It is based on an intelligent multi-agent based model, namely, iJADE, which supports various e-commerce applications. Using a gallery of 1200 images, our system shows an average segmentation rate of over 98%. iJADE Scene Segmentater segments faster and more powerfully than other contemporary image segmentation methods. The improvement in speed is more significant for large images.     

Sketch‐Based Editing Tools for Tumour Segmentation in 3D Medical Images

In the past years sophisticated automatic segmentation algorithms for various medical image segmentation problems have been developed. However, there are always cases where automatic algorithms fail to provide an acceptable segmentation. In these cases the user needs efficient segmentation editing tools, a problem which has not received much attention in research. We give a comprehensive overview on segmentation editing for three‐dimensional (3D) medical images. For segmentation editing in two‐dimensional (2D) images, we discuss a sketch‐based approach where the user modifies the segmentation in the contour domain. Based on this 2D interface, we present an image‐based as well as an image‐independent method for intuitive and efficient segmentation editing in 3D in the context of tumour segmentation in computed tomography (CT). Our editing tools have been evaluated on a database containing 1226 representative liver metastases, lung nodules and lymph nodes of different shape, size and image quality. In addition, we have performed a qualitative evaluation with radiologists and technical experts, proving the efficiency of our tools.     

基于区域中心的交互式图像前景提取方法

图像分割是从图像中提取有意义的区域,是图像处理和计算机视觉中的关键技术。而自动分割方法不能很好地处理前景复杂的图像,对此提出一种基于区域中心的交互式图像前景提取算法。针对图像前景的复杂度,很难用单一的相似区域描述前景,文中采用多个区域中心来刻画目标区域。为提升图像分割的稳定性,给出基于超像素颜色、空间位置和纹理信息的相似性度量方法;为确保图像分割区域的连通性和准确性,定义了基于超像素的测地距离计算方法。使用基于测地距离的超像素局部密度,来分析图像的若干区域中心;基于用户交互的方式来分析前景的区域中心,得到图像前景。经过大量彩色图像的仿真表明,在分割过程中利用少量的用户交互信息,可有效提升图像分割的稳定性和准确性。     

Object Repositioning Based on the Perspective in a Single Image

We propose an image editing system for repositioning objects in a single image based on the perspective of the scene. In our system, an input image is transformed into a layer structure that is composed of object layers and a background layer, and then the scene depth is computed from the ground region that is specified by the user using a simple boundary line. The object size and order of overlapping are automatically determined during the reposition based on the scene depth. In addition, our system enables the user to move shadows along with objects naturally by extracting the shadow mattes using only a few user‐specified scribbles. Finally, we demonstrate the versatility of our system through applications to depth‐of‐field effects, fog synthesis and 3D walkthrough in an image.     

Semantic Photo Synthesis

Composite images are synthesized from existing photographs by artists who make concept art, e.g., storyboards for movies or architectural planning. Current techniques allow an artist to fabricate such an image by digitally splicing parts of stock photographs. While these images serve mainly to “quickly”convey how a scene should look, their production is laborious. We propose a technique that allows a person to design a new photograph with substantially less effort. This paper presents a method that generates a composite image when a user types in nouns, such as “boat”and “sand.”The artist can optionally design an intended image by specifying other constraints. Our algorithm formulates the constraints as queries to search an automatically annotated image database. The desired photograph, not a collage, is then synthesized using graph‐cut optimization, optionally allowing for further user interaction to edit or choose among alternative generated photos. An implementation of our approach, shown in the associated video, demonstrates our contributions of (1) a method for creating specific images with minimal human effort, and (2) a combined algorithm for automatically building an image library with semantic annotations from any photo collection.     

多阈值优化交互式分割算法及其在医学图像中的应用

针对交互式图像分割方法对边界模糊的医学图像进行分割时通常需要用户标记较多的初始种子或进行二次交互的不足,提出了一种简化标记的多阈值优化交互式分割算法。该算法在GrowCut交互式算法基础上通过引入图像灰度直方图的多个阈值自动生成初始种子模板,并利用改进的细胞自动机迭代算法实现图像分割。算法简化了用户操作,提高了分割精度。应用该算法分别对临床100张肝脏图像和牙菌斑图像进行分割,结果显示了该算法的良好性能。     

结合KSW和FCNN的道路场景分割

目的 随着自动驾驶技术不断引入生活,机器视觉中道路场景分割算法的研究已至关重要。传统方法中大多数研究者使用机器学习方法对阈值分割,而近年来深度学习的引入,使得卷积神经网络被广泛应用于该领域。方法 针对传统阈值分割方法难以有效提取多场景下道路图像阈值的问题和直接用深度神经网络来训练数据导致过分割严重的问题,本文提出了结合KSW（key seat wiper）和全卷积神经网络（FCNN）的道路场景分割方法,该方法结合了KSW熵法及遗传算法,利用深度学习在不同场景下的特征提取,并将其运用到无人驾驶技术的道路分割中。首先对道路场景测试集利用KSW熵法及遗传算法得到训练集,然后导入到全卷积神经网络中进行训练得到有效训练模型,最后通过训练模型实现对任意一幅道路场景图分割。结果 实验结果表明,在KITTI数据集中进行测试,天空和树木的分割精度分别达到91.3%和94.3%,道路、车辆、行人的分割精度提高了2%左右。从分割结果中明显看出,道路图像中的积水、泥潭、树木等信息存在的过分割现象有良好的改观。结论 相比传统机器学习道路场景分割方法,本文方法在一定程度上提高了分割精度。对比深度学习直接应用于道路场景分割的方法,本文方法在一定程度上避免了过分割现象,提高了模型的鲁棒性。综上所述,本文提出的结合KSW和FCNN的道路场景分割算法有广泛的研究前景,有望应用于医学图像和遥感图像的处理中。