Reconstruction from Projections Using Grassmann Tensors

In this paper a general procedure is given for reconstruction of a set of feature points in an arbitrary dimensional projective space from their projections into lower dimensional spaces. This extends the methods applied in the well-studied problem of reconstruction of scene points in ℘³ given their projections in a set of images. In this case, the bifocal, trifocal and quadrifocal tensors are used to carry out this computation. It is shown that similar methods will apply in a much more general context, and hence may be applied to projections from ℘ ⁿ to ℘ ^m , which have been used in the analysis of dynamic scenes, and in radial distortion correction. For sufficiently many generic projections, reconstruction of the scene is shown to be unique up to projectivity, except in the case of projections onto one-dimensional image spaces (lines), in which case there are two solutions. Projections from ℘ ⁿ to ℘² have been considered by Wolf and Shashua (in International Journal of Computer Vision 48(1): 53–67, 2002), where they were applied to several different problems in dynamic scene analysis. They analyzed these projections using tensors, but no general way of defining such tensors, and computing the projections was given. This paper settles the general problem, showing that tensor definition and retrieval of the projections is always possible.     

Plenoptic Image Editing

This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate automatically to all other images as if the (unknown) 3D scene had itself been modified. The modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. The approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. The approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene's plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. A key element inrealizing these operations is a new volumetric decomposition technique for reconstructing an scene's plenoptic function from an incomplete set of camera viewpoints.     

Parameterized real-time moment computation on gray images using block techniques

Moments constitute a well-known tool in the field of image analysis and pattern recognition, but they suffer from the drawback of high computational cost. Efforts for the reduction of the required computational complexity have been reported, mainly focused on binary images, but recently some approaches for gray images have been also presented. In this study, we propose a simple but effective approach for the computation of gray image moments. The gray image is decomposed in a set of binary images. Some of these binary images are substituted by an ideal image, which is called “half-intensity” image. The remaining binary images are represented using the image block representation concept and their moments are computed fast using block techniques. The proposed method computes approximated moment values with an error of 2–3% from the exact values and operates in real time (i.e., video rate). The procedure is parameterized by the number m of “half-intensity” images used, which controls the approximation error and the speed gain of the method. The computational complexity is O(kL ²), where k is the number of blocks and L is the moment order.     

Undersampled Light Field Rendering by a Plane Sweep

Images synthesized by light field rendering exhibit aliasing artifacts when the light field is undersampled; adding new light field samples improves the image quality and reduces aliasing but new samples are expensive to acquire. Light field rays are traditionally gathered directly from the source images, but new rays can also be inferred through geometry estimation. This paper describes a light field rendering approach based on this principle that estimates geometry from the set of source images using multi‐baseline stereo reconstruction to supplement the existing light field rays to meet the minimum sampling requirement. The rendering and reconstruction steps are computed over a set of planes in the scene volume, and output images are synthesized by compositing results from these planes together. The planes are each processed independently and the number of planes can be adjusted to scale the amount of computation to achieve the desired frame rate. The reconstruction fidelity (and by extension image quality) is improved by a library of matching templates to support matches along discontinuities in the image or geometry (e.g. object profiles and concavities). Given a set of silhouette images, the visual hull can be constructed and applied to further improve reconstruction by removing outlier matches. The algorithm is efficiently implemented by a set of image filter operations on commodity graphics hardware and achieves image synthesis at interactive rates.     

可见光与红外图像融合质量评价指标分析

目的 客观评价作为图像融合的重要研究领域,是评价融合算法性能的有力工具。目前,已有几十种不同类型的评价指标,但各应用领域包括可见光与红外图像融合,仍缺少统一的选择依据。为了方便比较不同融合算法性能,提出一种客观评价指标的通用分析方法并应用于可见光与红外图像融合。方法 将可见光与红外图像基准数据集中的客观评价指标分为两类,分别是基于融合图像的评价指标与基于源图像和融合图像的评价指标。采用Kendall相关系数分析融合指标间的相关性,聚类得到指标分组;采用Borda计数排序法统计算法的综合排序,分析单一指标排序和综合排序的相关性,得到一致性较高的指标集合;采用离散系数分析指标均值随不同算法的波动程度,选择充分体现不同算法间差异的指标;综合相关性分析、一致性分析及离散系数分析,总结具有代表性的建议指标集合。结果 在13对彩色可见光与红外和8对灰度可见光与红外两组图像源中,分别统计分析不同图像融合算法的客观评价数据,得到可见光与红外图像融合的建议指标集(标准差、边缘保持度),作为融合算法性能评估的重要参考。相较于现有方法,实验覆盖20种融合算法和13种客观评价指标,并且不依赖主观评价结果。结论...     

The conformal alpha shape filtration

Conformal alpha shapes are a new filtration of the Delaunay triangulation of a finite set of points in ℝ^d. In contrast to (ordinary) alpha shapes the new filtration is parameterized by a local scale parameter instead of the global scale parameter in alpha shapes. The local scale parameter conforms to the local geometry and is motivated from applications and previous algorithms in surface reconstruction. We show how conformal alpha shapes can be used for surface reconstruction of non-uniformly sampled surfaces, which is not possible with alpha shapes.     

Gamut Constrained Illuminant Estimation

This paper presents a novel solution to the illuminant estimation problem: the problem of how, given an image of a scene taken under an unknown illuminant, we can recover an estimate of that light. The work is founded on previous gamut mapping solutions to the problem which solve for a scene illuminant by determining the set of diagonal mappings which take image data captured under an unknown light to a gamut of reference colours taken under a known light. Unfortunately, a diagonal model is not always a valid model of illumination change and so previous approaches sometimes return a null solution. In addition, previous methods are difficult to implement. We address these problems by recasting the problem as one of illuminant classification: we define a priori a set of plausible lights thus ensuring that a scene illuminant estimate will always be found. A plausible light is represented by the gamut of colours observable under it and the illuminant in an image is classified by determining the plausible light whose gamut is most consistent with the image data. We show that this step (the main computational burden of the algorithm) can be performed simply and efficiently by means of a non-negative least-squares optimisation. We report results on a large set of real images which show that it provides excellent illuminant estimation, outperforming previous algorithms. First online version published in February, 2006     

Tight lower bounds for certain parameterized NP-hard problems

Based on the framework of parameterized complexity theory, we derive tight lower bounds on the computational complexity for a number of well-known NP-hard problems. We start by proving a general result, namely that the parameterized weighted satisfiability problem on depth-t circuits cannot be solved in time n^o(k)m^O(1), where n is the circuit input length, m is the circuit size, and k is the parameter, unless the (t − 1)-st level W[t − 1] of the W-hierarchy collapses to FPT. By refining this technique, we prove that a group of parameterized NP-hard problems, including weighted sat, hitting set, set cover, and feature set, cannot be solved in time n^o(k)m^O(1), where n is the size of the universal set from which the k elements are to be selected and m is the instance size, unless the first level W[1] of the W-hierarchy collapses to FPT. We also prove that another group of parameterized problems which includes weighted q-sat (for any fixed q 2), clique, independent set, and dominating set, cannot be solved in time n^o(k) unless all search problems in the syntactic class SNP, introduced by Papadimitriou and Yannakakis, are solvable in subexponential time. Note that all these parameterized problems have trivial algorithms of running time either n^km^O(1) or O(n^k).     

Using vanishing points for camera calibration and coarse 3D reconstruction from a single image

In this paper, we show how to calibrate a camera and to recover the geometry and the photometry (textures) of objects from a single image. The aim of this work is to make it possible walkthrough and augment reality in a 3D model reconstructed from a single image. The calibration step does not need any calibration target and makes only four assumptions: (1) the single image contains at least two vanishing points, (2) the length (in 3D space) of one line segment (for determining the translation vector) in the image is known, (3) the principle point is the center of the image, and (4) the aspect ratio is fixed by the user. Each vanishing point is determined from a set of parallel lines. These vanishing points help determine a 3D world coordinate system R _o. After having computed the focal length, the rotation matrix and the translation vector are evaluated in turn for describing the rigid motion between R _o and the camera coordinate system R _c. Next, the reconstruction step consists in placing, rotating, scaling, and translating a rectangular 3D box that must fit at best with the potential objects within the scene as seen through the single image. With each face of a rectangular box, a texture that may contain holes due to invisible parts of certain objects is assigned. We show how the textures are extracted and how these holes are located and filled. Our method has been applied to various real images (pictures scanned from books, photographs) and synthetic images.     

局部均值噪声估计的盲3维滤波降噪算法

目的 图像在获取和传输的过程中很容易受到噪声的干扰,图像降噪作为众多图像处理系统的预处理模块在过去数十年中得到了广泛的研究。在已提出的降噪算法中,往往采用加性高斯白噪声模型AWGN（additive white Gaussian noise）为噪声建模,噪声水平（严重程度）由方差参数控制。经典的BM3D 3维滤波算法属于非盲降噪（non-blind denoising algorithm）算法,在实际使用中需要由人工评估图像噪声水平并设置参数,存在着噪声评估值随机性大而导致无法获得最佳降噪效果的问题。为此,提出了一种新的局部均值噪声估计（LME）算法并作为BM3D算法的前置预处理模块。方法 本文专注于利用基于自然统计规律（NSS）的图像质量感知特征和局部均值估计技术构建图像噪声水平预测器,并通过它高效地获得噪声图像中准确的噪声水平值。关于自然场景统计方面的研究表明,无失真的自然场景图像在空域或者频率域上具有显著的统计规律,一旦受到噪声干扰会产生规律性的偏移,可以提取这些特征值作为反映图像质量好坏的图像质量感知特征。另外,局部均值估计因其简单而高效率的预测特性被采用。具体实现上,在具有广泛代表性且未受噪声干扰图像集合上添加不同噪声水平的高斯噪声构建失真图像集合,然后利用小波变换对这些失真图像进行不同尺度和不同方向的分解,再用广义高斯分布模型（GGD）提取子带滤波系数的统计信息构成描述图像失真程度的特征矢量,最后用每幅失真图像上所提取的特征矢量及对其所施加的高斯噪声水平值构成了失真特征矢量库。在降噪阶段,用相同的特征提取方法提取待降噪的图像的特征矢量并在失真特征矢量库中检索出与之类似的若干特征矢量及它们所对应的噪声水平值,然后用局部均值法估计出待降噪图像中高斯噪声大小作为经典BM3D算法的输入参数。结果 改进后的BM3D算法转换为盲降噪算法,称为BM3D-LME（block-matching and 3D filtering based on local means estimation）算法。准确的噪声估计对于诸如图像降噪,图像超分辨率和图像分割等图像处理任务非常重要。已经验证了所提出噪声水平估计算法的准确性、鲁棒性和有效性。结论 相对人工进行噪声估计,LME算法能够准确、快速地估算出任意待降噪图像中的噪声大小。配合BM3D算法使用后,有效提高了它的实际降噪效果并扩大它的应用范围。     

各向异性导向滤波的红外与可见光图像融合

目的 针对红外与可见光图像融合时易产生边缘细节信息丢失、融合结果有光晕伪影等问题，同时为充分获取多源图像的重要特征，将各向异性导向滤波和相位一致性结合，提出一种红外与可见光图像融合算法。方法 首先，采用各向异性导向滤波从源图像获得包含大尺度变化的基础图和包含小尺度细节的系列细节图；其次，利用相位一致性和高斯滤波计算显著图，进而通过对比像素显著性得到初始权重二值图，再利用各向异性导向滤波优化权重图，达到去除噪声和抑制光晕伪影；最后，通过图像重构得到融合结果。结果 从主客观两个方面，将所提方法与卷积神经网络（convolutional neural network，CNN）、双树复小波变换（dual-tree complex wavelet transform，DTCWT）、导向滤波（guided filtering，GFF）和各向异性扩散（anisotropic diffusion，ADF）等4种经典红外与可见光融合方法在TNO公开数据集上进行实验对比。主观分析上，所提算法结果在边缘细节、背景保存和目标完整度等方面均优于其他4种方法；客观分析上，选取互信息（mutual information，MI）、边缘信息保持度（degree of edge information，Q^AB/F）、熵（entropy，EN）和基于梯度的特征互信息（gradient based feature mutual information，FMI_gradient）等4种图像质量评价指数进行综合评价。相较于其他4种方法，本文算法的各项指标均有一定幅度的提高，MI平均值较GFF提高了21.67%，Q^AB/F平均值较CNN提高了20.21%，EN平均值较CNN提高了5.69%，FMI_gradient平均值较GFF提高了3.14%。结论 本文基于各向异性导向滤波融合算法可解决原始导向滤波存在的细节"光晕"问题，有效抑制融合结果中伪影的产生，同时具有尺度感知特性，能更好保留源图像的边缘细节信息和背景信息，提高了融合结果的准确性。     

Three-dimensional scene flow

Just as optical flow is the two-dimensional motion of points in an image, scene flow is the three-dimensional motion of points in the world. The fundamental difficulty with optical flow is that only the normal flow can be computed directly from the image measurements, without some form of smoothing or regularization. In this paper, we begin by showing that the same fundamental limitation applies to scene flow; however, many cameras are used to image the scene. There are then two choices when computing scene flow: 1) perform the regularization in the images or 2) perform the regularization on the surface of the object in the scene. In this paper, we choose to compute scene flow using regularization in the images. We describe three algorithms, the first two for computing scene flow from optical flows and the third for constraining scene structure from the inconsistencies in multiple optical flows.     

Unsupervised scene segmentation using sparse coding context

This paper presents an approach to image understanding on the aspect of unsupervised scene segmentation. With the goal of image understanding in mind, we consider ‘unsupervised scene segmentation’ a task of dividing a given image into semantically meaningful regions without using annotation or other human-labeled information. We seek to investigate how well an algorithm can achieve at partitioning an image with limited human-involved learning procedures. Specifically, we are interested in developing an unsupervised segmentation algorithm that only relies on the contextual prior learned from a set of images. Our algorithm incorporates a small set of images that are similar to the input image in their scene structures. We use the sparse coding technique to analyze the appearance of this set of images; the effectiveness of sparse coding allows us to derive a priori the context of the scene from the set of images. Gaussian mixture models can then be constructed for different parts of the input image based on the sparse-coding contextual prior, and can be combined into an Markov-random-field-based segmentation process. The experimental results show that our unsupervised segmentation algorithm is able to partition an image into semantic regions, such as buildings, roads, trees, and skies, without using human-annotated information. The semantic regions generated by our algorithm can be useful, as pre-processed inputs for subsequent classification-based labeling algorithms, in achieving automatic scene annotation and scene parsing.     

Shape and motion from image streams under orthography: a factorization method

Inferring scene geometry and camera motion from a stream of images is possible in principle, but is an ill-conditioned problem when the objects are distant with respect to their size. We have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step.An image stream can be represented by the 2F×P measurement matrix of the image coordinates of P points tracked through F frames. We show that under orthographic projection this matrix is of rank 3.Based on this observation, the factorization method uses the singular-value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively. Two of the three translation components are computed in a preprocessing stage. The method can also handle and obtain a full solution from a partially filled-in measurement matrix that may result from occlusions or tracking failures.The method gives accurate results, and does not introduce smoothing in either shape or motion. We demonstrate this with a series of experiments on laboratory and outdoor image streams, with and without occlusions.     

Multiple Contour Finding and Perceptual Grouping using Minimal Paths

We address the problem of finding a set of contour curves in an image. We consider the problem of perceptual grouping and contour completion, where the data is a set of points in the image. A new method to find complete curves from a set of contours or edge points is presented. Our approach is based on a previous work on finding contours as minimal paths between two end points using the fast marching algorithm (L. D Cohen and R. Kimmel, International Journal of Computer Vision, Vol. 24, No. 1, pp. 57–78, 1997). Given a set of key points, we find the pairs of points that have to be linked and the paths that join them. We use the saddle points of the minimal action map. The paths are obtained by backpropagation from the saddle points to both points of each pair.In a second part, we propose a scheme that does not need key points for initialization. A set of key points is automatically selected from a larger set of admissible points. At the same time, saddle points between pairs of key points are extracted. Next, paths are drawn on the image and give the minimal paths between selected pairs of points. The set of minimal paths completes the initial set of contours and allows to close them. We illustrate the capability of our approach to close contours with examples on various images of sets of edge points of shapes with missing contours.     

Parameterized Proof Complexity

We propose a proof-theoretic approach for gaining evidence that certain parameterized problems are not fixed-parameter tractable. We consider proofs that witness that a given propositional formula cannot be satisfied by a truth assignment that sets at most k variables to true, considering k as the parameter (we call such a formula a parameterized contradiction). One could separate the parameterized complexity classes FPT and W[SAT] by showing that there is no fpt-bounded parameterized proof system for parameterized contradictions, i.e., that there is no proof system that admits proofs of size f(k)n ^O(1) where f is a computable function and n represents the size of the propositional formula. By way of a first step, we introduce the system of parameterized tree-like resolution and show that this system is not fpt-bounded. Indeed, we give a general result on the size of shortest tree-like resolution proofs of parameterized contradictions that uniformly encode first-order principles over a universe of size n. We establish a dichotomy theorem that splits the exponential case of Riis’s complexity gap theorem into two subcases, one that admits proofs of size f(k)n ^O(1) and one that does not. We also discuss how the set of parameterized contradictions may be embedded into the set of (ordinary) contradictions by the addition of new axioms. When embedded into general (DAG-like) resolution, we demonstrate that the pigeonhole principle has a proof of size 2 ^k n ². This contrasts with the case of tree-like resolution where the embedded pigeonhole principle falls into the “non-FPT” category of our dichotomy.     

Multi-Frame Correspondence Estimation Using Subspace Constraints

When a rigid scene is imaged by a moving camera, the set of all displacements of all points across multiple frames often resides in a low-dimensional linear subspace. Linear subspace constraints have been used successfully in the past for recovering 3D structure and 3D motion information from multiple frames (e.g., by using the factorization method of Tomasi and Kanade (1992, International Journal of Computer Vision, 9:137–154)). These methods assume that the 2D correspondences have been precomputed. However, correspondence estimation is a fundamental problem in motion analysis. In this paper we show how the multi-frame subspace constraints can be used for constraining the 2D correspondence estimation process itself.We show that the multi-frame subspace constraints are valid not only for affine cameras, but also for a variety of imaging models, scene models, and motion models. The multi-frame subspace constraints are first translated from constraints on correspondences to constraints directly on image measurements (e.g., image brightness quantities). These brightness-based subspace constraints are then used for estimating the correspondences, by requiring that all corresponding points across all video frames reside in the appropriate low-dimensional linear subspace.The multi-frame subspace constraints are geometrically meaningful, and are {not} violated at depth discontinuities, nor when the camera-motion changes abruptly. These constraints can therefore replace {heuristic} constraints commonly used in optical-flow estimation, such as spatial or temporal smoothness.     

Illumination of image-based objects

A new data representation of image-based objects is presented. With this representation, the user can change the illumination as well as the viewpoint of an image-based scene. Physically correct imagery can be generated without knowing any geometrical information (e.g. depth or surface normal) of the scene. By treating each pixel on the image plane as a surface element, we can measure its apparent BRDF (bidirectional reflectance distribution function) by collecting information in the sampled images. These BRDFs allow us to calculate the correct pixel colour under a new illumination set-up by fitting the intensity, direction and number of the light sources. We demonstrate that the proposed representation allows re-rendering of the scene illuminated by different types of light sources. Moreover, two compression schemes, spherical harmonics and discrete cosine transform, are proposed to compress the huge amount of tabular BRDF data. © 1998 John Wiley & Sons, Ltd.     

利用运动线索的单目深度测量

目的 传统的单目视觉深度测量方法具有设备简单、价格低廉、运算速度快等优点,但需要对相机进行复杂标定,并且只在特定的场景条件下适用。为此,提出基于运动视差线索的物体深度测量方法,从图像中提取特征点,利用特征点与图像深度的关系得到测量结果。方法 对两幅图像进行分割,获取被测量物体所在区域;然后采用本文提出的改进的尺度不变特征变换SIFT（scale-invariant feature transtorm）算法对两幅图像进行匹配,结合图像匹配和图像分割的结果获取被测量物体的匹配结果;用Graham扫描法求得匹配后特征点的凸包,获取凸包上最长线段的长度;最后利用相机成像的基本原理和三角几何知识求出图像深度。结果 实验结果表明,本文方法在测量精度和实时性两方面都有所提升。当图像中的物体不被遮挡时,实际距离与测量距离之间的误差为2.60%,测量距离的时间消耗为1.577 s;当图像中的物体存在部分遮挡时,该方法也获得了较好的测量结果,实际距离与测量距离之间的误差为3.19%,测量距离所需时间为1.689 s。结论 利用两幅图像上的特征点来估计图像深度,对图像中物体存在部分遮挡情况具有良好的鲁棒性,同时避免了复杂的摄像机标定过程,具有实际应用价值。