Segmentation and Interpretation of Multicolored Objects with Highlights

This paper presents a segmentation system, based on a general framework for segmentation, that returns not only regions that correspond to coherent surfaces in an image, but also low-level interpretations of those regions' physical characteristics. This system is valid for images of piecewise uniform dielectric objects with highlights, moving it beyond the capabilities of previous physics-based segmentation algorithms which assume uniformly colored objects. This paper presents a summary of the complete system and focuses on two extensions of it that demonstrate its interpretive capacity and applicability to more complex scenes. The first extension provides interpretations of a scene by reasoning about the likelihood of different physical characteristics of simple image regions. The second extension allows the system to handle highlights within the general framework for segmentation. The resulting segmentations and interpretations more closely match our perceptions of objects since the resulting regions correspond to coherent surfaces, even when those surfaces have multiple colors and highlights.     

Contrast restoration of weather degraded images

Images of outdoor scenes captured in bad weather suffer from poor contrast. Under bad weather conditions, the light reaching a camera is severely scattered by the atmosphere. The resulting decay in contrast varies across the scene and is exponential in the depths of scene points. Therefore, traditional space invariant image processing techniques are not sufficient to remove weather effects from images. We present a physics-based model that describes the appearances of scenes in uniform bad weather conditions. Changes in intensities of scene points under different weather conditions provide simple constraints to detect depth discontinuities in the scene and also to compute scene structure. Then, a fast algorithm to restore scene contrast is presented. In contrast to previous techniques, our weather removal algorithm does not require any a priori scene structure, distributions of scene reflectances, or detailed knowledge about the particular weather condition. All the methods described in this paper are effective under a wide range of weather conditions including haze, mist, fog, and conditions arising due to other aerosols. Further, our methods can be applied to gray scale, RGB color, multispectral and even IR images. We also extend our techniques to restore contrast of scenes with moving objects, captured using a video camera.     

Evidence-based recognition of 3-D objects

An evidence-based recognition technique is defined that identifies 3-D objects by looking for their notable features. This technique makes use of an evidence rule base, which is a set of salient or evidence conditions with corresponding evidence weights for various objects in the database. A measure of similarity between the set of observed features and the set of evidence conditions for a given object in the database is used to determine the identity of an object in the scene or reject the object(s) in the scene as unknown. This procedure has polynomial time complexity and correctly identifies a variety of objects in both synthetic and real range images. A technique for automatically deriving the evidence rule base from training views of objects is shown to generate evidence conditions that successfully identify new views of those objects     

Image completion using prediction concept via support vector regression

Image completion is a widely used method for automatically removing objects or repairing the damaged portions of an image. However, information of the original image is often lacking in reconstructed structures; therefore, images with complex structures are difficult to restore. This study proposes a prediction-oriented image completion mechanism (PICM), which applies the prediction concept to image completion using numerous techniques and methods. The experiment results indicate that under normal circumstances, our PICM not only produces good inpainting quality but it is also easy to use.     

Image/source statistics of surfaces in natural scenes

Perceiving surfaces in a manner that accords with their physical properties is essential for successful behaviour. Since, however, a given retinal image can have been generated by an infinite variety of natural surfaces with different geometrical and/or physical qualities, the corresponding percepts cannot be determined by the stimulus per se. Rather, resolution of this quandary requires a strategy of vision that incorporates the statistical relationship of the information in retinal images to its sources in representative environments. To examine this probabilistic relationship with respect to the features of object surfaces, we analysed a database of range images in which the distances of all the objects in a series of natural scenes were measured with respect to the image plane by a laser range scanner. By taking any particular scene obtained in this way to be made up of a set of concatenated surface patches, we were able to explore the statistics of scene roughness, size-distance relationships, surface orientation and local curvature, as well as the independent components of natural surfaces. The relevance of these statistics to both perception and the neuronal organization of the underlying visual circuitry is discussed.     

Parameter estimation in Markov random field contextual models usinggeometric models of objects

We present a new scheme for the estimation of Markov random field line process parameters which uses geometric CAD models of the objects in the scene. The models are used to generate synthetic images of the objects from random view points. The edge maps computed from the synthesized images are used as training samples to estimate the line process parameters using a least squares method. We show that this parameter estimation method is useful for detecting edges in range as well as intensity edges. The main contributions of the paper are: 1) use of CAD models to obtain true edge labels which are otherwise not available; and 2) use of canonical Markov random field representation to reduce the number of parameters     

The recovery of three-dimensional structure from image curves

A class of inferences is described which allows the recovery of three-dimensional structures from the two-dimensional curves in an image. Unlike most previous methods, these inferences do not require restrictive assumptions or prior knowledge regarding the scene. They are based on the assumption that the camera viewpoint and the positions of the illumination sources are independent of the objects in the scene. From these independence assumptions, it can be shown that many potential interpretations of image curves are highly improbable. By eliminating these improbable interpretations it is possible to segment the image into sets of related image features and derive many three-space relations.     

图注意力网络的场景图到图像生成模型

目的 目前文本到图像的生成模型仅在具有单个对象的图像数据集上表现良好，当一幅图像涉及多个对象和关系时，生成的图像就会变得混乱。已有的解决方案是将文本描述转换为更能表示图像中场景关系的场景图结构，然后利用场景图生成图像，但是现有的场景图到图像的生成模型最终生成的图像不够清晰，对象细节不足。为此，提出一种基于图注意力网络的场景图到图像的生成模型，生成更高质量的图像。方法 模型由提取场景图特征的图注意力网络、合成场景布局的对象布局网络、将场景布局转换为生成图像的级联细化网络以及提高生成图像质量的鉴别器网络组成。图注意力网络将得到的具有更强表达能力的输出对象特征向量传递给改进的对象布局网络，合成更接近真实标签的场景布局。同时，提出使用特征匹配的方式计算图像损失，使得最终生成图像与真实图像在语义上更加相似。结果 通过在包含多个对象的COCO-Stuff图像数据集中训练模型生成64×64像素的图像，本文模型可以生成包含多个对象和关系的复杂场景图像，且生成图像的Inception Score为7.8左右，与原有的场景图到图像生成模型相比提高了0.5。结论 本文提出的基于图注意力网络的场景图到图像生成模型不仅可以生成包含多个对象和关系的复杂场景图像，而且生成图像质量更高，细节更清晰。     

Robust PCA-based solution to image composition using augmented Lagrange multiplier (ALM)

Computational photography relies on specialized image-processing techniques to combine multiple images captured by a camera to generate a desired image of the scene. We first consider the high dynamic range (HDR) imaging problem. We can change either the exposure time or the aperture while capturing multiple images of the scene to generate an HDR image. This paper addresses the HDR imaging problem for static and dynamic scenes captured using a stationary camera under various aperture and exposure settings, when we do not have any knowledge of the camera settings. We have proposed a novel framework based on sparse representation which enables us to process images while getting rid of artifacts due to moving objects and defocus blur. We show that the proposed approach is able to produce significantly good results through dynamic object rejection and deblurring capabilities. We compare the results with other competitive approaches and discuss the relative advantages of the proposed approach.     

A multi-temporal masking classification method for vineyard monitoring in central Spain

Abstract

The objective of image segmentation in remote sensing is to define regions in an image that correspond to objects in the ground scene. Traditional scene models underlying image segmentation procedures have assumed that objects as manifest in images have internal variances that are both low and equal. This scene model is unrealistically simple. An alternative scene model recognizes different scales of objects in scenes. Each level in the hierarchy is nested, or composed of objects or categories of objects from the preceding level. Different objects may have distinct attributes, allowing for relaxation of assumptions like equal variance.

A multiple-pass, region-based segmentation algorithm improves the segmentation of images from scenes better modelled as a nested hierarchy. A multiple-pass approach allows slow and careful growth of regions while inter-region distances are below a global threshold. Past the global threshold, a minimum region size parameter forces development of regions in areas of high local variance. Maximum and viable region size parameters limit the development of undesirably large regions.

Application of the segmentation algorithm for forest stand delineation in Landsat TM imagery yields regions corresponding to identifiable features in the landscape. The use of a local variance, adaptive-window texture channel in conjunction with spectral bands improves the ability to define regions corresponding to sparsely-stocked forest stands which have high internal variance.