Projective Reconstruction from Multiple Views with Minimization of 2D Reprojection Error

The problem of projective reconstruction by minimization of the 2D reprojection error in multiple images is considered. Although bundle adjustment techniques can be used to minimize the 2D reprojection error, these methods being based on nonlinear optimization algorithms require a good starting point. Quasi-linear algorithms with better global convergence properties can be used to generate an initial solution before submitting it to bundle adjustment for refinement. In this paper, we propose a factorization-based method to integrate the initial search as well as the bundle adjustment into a single algorithm consisting of a sequence of weighted least-squares problems, in which a control parameter is initially set to a relaxed state to allow the search of a good initial solution, and subsequently tightened up to force the final solution to approach a minimum point of the 2D reprojection error. The proposed algorithm is guaranteed to converge. Our method readily handles images with missing points.     

Sketch‐Based Editing Tools for Tumour Segmentation in 3D Medical Images

In the past years sophisticated automatic segmentation algorithms for various medical image segmentation problems have been developed. However, there are always cases where automatic algorithms fail to provide an acceptable segmentation. In these cases the user needs efficient segmentation editing tools, a problem which has not received much attention in research. We give a comprehensive overview on segmentation editing for three‐dimensional (3D) medical images. For segmentation editing in two‐dimensional (2D) images, we discuss a sketch‐based approach where the user modifies the segmentation in the contour domain. Based on this 2D interface, we present an image‐based as well as an image‐independent method for intuitive and efficient segmentation editing in 3D in the context of tumour segmentation in computed tomography (CT). Our editing tools have been evaluated on a database containing 1226 representative liver metastases, lung nodules and lymph nodes of different shape, size and image quality. In addition, we have performed a qualitative evaluation with radiologists and technical experts, proving the efficiency of our tools.     

An evolutionary tabu search for cell image segmentation

Many engineering problems can be formulated as optimization problems. It has become more and more important to develop an efficient global optimization technique for solving these problems. In this paper, we propose an evolutionary tabu search (ETS) for cell image segmentation. The advantages of genetic algorithms (GA) and TS algorithms are incorporated into the proposed method. More precisely, we incorporate "the survival of the fittest" from evolutionary algorithms into TS. The method has been applied to the segmentation of several kinds of cell images. The experimental results show that the new algorithm is a practical and effective one for global optimization; it can yield good, near-optimal solutions and has better convergence and robustness than other global optimization approaches.     

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a realistic modification of its 3D model. In this paper, we show that such information can be learned from user‐clicked 2D images and a template 3D model of the target animal. We present a volumetric deformation framework that produces a set of new 3D models by deforming a template 3D model according to a set of user‐clicked images. Our framework is based on a novel locally‐bounded deformation energy, where every local region has its own stiffness value that bounds how much distortion is allowed at that location. We jointly learn the local stiffness bounds as we deform the template 3D mesh to match each user‐clicked image. We show that this seemingly complex task can be solved as a sequence of convex optimization problems. We demonstrate the effectiveness of our approach on cats and horses, which are highly deformable and articulated animals. Our framework produces new 3D models of animals that are significantly more plausible than methods without learned stiffness.     

On partitions and presortedness of sequences

To take advantage of existing order in a sequence when sorting, we evaluate the quantity of this information by the minimal size of decomposition of the input sequence, particularly the minimal size of chain and of monotonic partitions. Some sorting strategies that are optimal with respect to these measures of presortedness are presented. The relationships between these new measures of presortedness and other known measures have also been explored. As an application, we carry out the optimality of an adaptive sorting algorithm Skiena'sMelsort. For some special partitioning strategies, we present two sorting algorithms based on Dijkstra'sSmoothsort. Moreover, the optimalities of these two algorithms are demonstrated. By examining the optimalities of sorting algorithms with respect to certain measures of presortedness, we also propose some optimal sorting strategies for one class of measures. Finally, we discuss other types of sorting problems, such as sorting multisets and topological sorting. In particular, we derive an optimal algorithm for sorting multisets and for finding the multiset sizes as well.A preliminary version of this paper was presented at the Second Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, USA (January 28–30, 1991).The main part of the work was conducted while the author was with Lund University.     

Estimating 3D shape from degenerate sequences with missing data

Reconstructing a 3D scene from a moving camera is one of the most important issues in the field of computer vision. In this scenario, not all points are known in all images (e.g. due to occlusion), thus generating missing data. On the other hand, successful 3D reconstruction algorithms like Tomasi & Kanade’s factorization method, require an orthographic model for the data, which is adequate in close-up views. The state-of-the-art handles the missing points in this context by enforcing rank constraints on the point track matrix. However, quite frequently, close-up views tend to capture planar surfaces producing degenerate data. Estimating missing data using the rank constraint requires that all known measurements are “full rank” in all images of the sequence. If one single frame is degenerate, the whole sequence will produce high errors on the reconstructed shape, even though the observation matrix verifies the rank 4 constraint. In this paper, we propose to solve the structure from motion problem with degenerate data, introducing a new factorization algorithm that imposes the full scaled-orthographic model in one single optimization procedure. By imposing all model constraints, a unique (correct) 3D shape is estimated regardless of the data degeneracies. Experiments show that remarkably good reconstructions are obtained with an approximate models such as orthography.     

Extracting View-Dependent Depth Maps from a Collection of Images

Stereo correspondence algorithms typically produce a single depth map. In addition to the usual problems of occlusions and textureless regions, such algorithms cannot model the variation in scene or object appearance with respect to the viewing position. In this paper, we propose a new representation that overcomes the appearance variation problem associated with an image sequence. Rather than estimating a single depth map, we associate a depth map with each input image (or a subset of them). Our representation is motivated by applications such as view interpolation and depth-based segmentation for model-building or layer extraction. We describe two approaches to extract such a representation from a sequence of images.The first approach, which is more classical, computes the local depth map associated with each chosen reference frame independently. The novelty of this approach lies in its combination of shiftable windows, temporal selection, and graph cut optimization. The second approach simultaneously optimizes a set of self-consistent depth maps at multiple key-frames. Since multiple depth maps are estimated simultaneously, visibility can be modeled explicitly and disparity consistency imposed across the different depth maps. Results, which include a difficult specular scene example, show the effectiveness of our approach.     

Hybrid Parallel Bundle Adjustment for 3D Scene Reconstruction with Massive Points

Bundle adjustment(BA) is a crucial but time consuming step in 3D reconstruction.In this paper,we intend to tackle a special class of BA problems where the reconstructed 3D points are much more numerous than the camera parameters,called Massive-Points BA(MPBA) problems.This is often the case when high-resolution images are used.We present a design and implementation of a new bundle adjustment algorithm for efficiently solving the MPBA problems.The use of hardware parallelism,the multi-core CPUs as well as GPUs,is explored.By careful memory-usage design,the graphic-memory limitation is effectively alleviated.Several modern acceleration strategies for bundle adjustment,such as the mixed-precision arithmetics,the embedded point iteration,and the preconditioned conjugate gradients,are explored and compared.By using several high-resolution image datasets,we generate a variety of MPBA problems,with which the performance of five bundle adjustment algorithms are evaluated.The experimental results show that our algorithm is up to 40 times faster than classical Sparse Bundle Adjustment,while maintaining comparable precision.     

An introduction to nonlinear programming—IV Numerical methods for constrained minimization

In this paper, we consider the problem of determining the numerical solution of constrained minimization problems. This discussion complements the theoretical development regarding the nature of the solution of the general nonlinear programming problem that was presented in Part I of this series of articles. As in our earlier discussion, the objective of this article is to review basic ideas and to illustrate the application of the ideas by describing specific computational algorithms. Thus, we discuss a variety of algorithms but not always in their greatest detail. References are provided for more detailed expositions and for generalizations and extensions of the basic algorithms. Attention is given first to the solution of problems with linear constraints. Then, approximation methods that reduce nonlinearly constrained problems to a sequence of linear programming problems are described. Finally, the discussion is completed by describing methods that reduce nonlinearly constrained problems to a sequence of unconstrained problems. Much of the discussion of specific algorithms draws upon the results presented in Parts II and III of this series.     

基于ShuffleNet V2算法的三维视线估计

为了解决当前视线估计网络复杂度较深、精度不高的问题,同时为了未来将网络部署在移动设备端,提出了一种基于ShuffleNet V2算法的视线估计网络,其由脸部和眼睛两个子网络构成。脸部子网络通过ResNetV2网络对脸部图片进行特征处理,并加入人脸对齐算法,减少头部角度误差的影响。眼睛子网络通过ShuffleNet V2...     

Generalised residual images’ effect on illumination artifact removal for correspondence algorithms

The intensity (grey value) consistency of image pixels in a sequence or stereo camera setup is of central importance to numerous computer vision applications. Most stereo matching and optical flow algorithms minimise an energy function composed of a data term and a regularity or smoothing term. To date, well performing methods rely on the intensity consistency of the image pixel values to model the data term. Such a simple model fails if the illumination is (even slightly) different between the input images. Amongst other situations, this may happen due to background illumination change over the sequence, different reflectivity of a surface, vignetting, or shading effects.In this paper, we investigate the removal of illumination artifacts and show that generalised residual images substantially improve the accuracy of correspondence algorithms. In particular, we motivate the concept of residual images and show two evaluation approaches using either ground truth correspondence fields (for stereo matching and optical flow algorithms) or errors based on a predicted view (for stereo matching algorithms).     

Toward Compression of Encrypted Images and Video Sequences

We present a framework for compressing encrypted media, such as images and videos. Encryption masks the source, rendering traditional compression algorithms ineffective. By conceiving of the problem as one of distributed source coding, it has been shown in prior work that encrypted data are as compressible as unencrypted data. However, there are two major challenges to realize these theoretical results. The first is the development of models that capture the underlying statistical structure and are compatible with our framework. The second is that since the source is masked by encryption, the compressor does not know what rate to target. We tackle these issues in this paper. We first develop statistical models for images before extending it to videos, where our techniques really gain traction. As an illustration, we compare our results to a state-of-the-art motion-compensated lossless video encoder that requires unencrypted video input. The latter compresses each unencrypted frame of the “Foreman” test sequence by 59% on average. In comparison, our proof-of-concept implementation, working on encrypted data, compresses the same sequence by 33%. Next, we develop and present an adaptive protocol for universal compression and show that it converges to the entropy rate. Finally, we demonstrate a complete implementation for encrypted video.      

A comparative study of staff removal algorithms

This paper presents a quantitative comparison of different algorithms for the removal of stafflines from music images. It contains a survey of previously proposed algorithms and suggests a new skeletonization based approach. We define three different error metrics, compare the algorithms with respect to these metrics and measure their robustness with respect to certain image defects. Our test images are computer-generated scores on which we apply various image deformations typically found in real-world data. In addition to modern western music notation our test set also includes historic music notation such as mensural notation and lute tablature. Our general approach and evaluation methodology is not specific to staff removal, but applicable to other segmentation problems as well.     

基于自适应权值的点云三维物体重建算法研究

基于三维扫描点云数据的三维物体重建是计算机图形学中非常重要的课题,在计 算机动画、医学图像处理等多方面都有应用。其中基于最小二乘问题的Levenberg-Marquart 算 法和基于极大似然估计的M-Estimator 算法都是不错的方案。但是当点的数量过多过少或者点 云中有噪声时,这些方案产生的结果都会有较大的误差,影响重建的效果。为了解决这两个问 题,结合Levenberg-Marquart 算法和M-Estimator 算法,提出了一种新的算法。该算法结合 Levenberg-Marquart 算法较快的收敛性和M-Estimator 算法的抗噪性,能很好地解决点数量较多 和噪声点影响结果的问题。通过在M-Estimator 的权重函数上进行改进,提出自适应的权值函 数,用灵活变动和自适应的值代替原来的固定值,使算法在噪声等级较高时也能表现良好。最 后将算法应用在球体和圆柱上,并和最新的研究成果进行对比,数据说明算法无论是在点云数 量较多还是在噪声等级较高的情况下都明显优于其他已知算法。     

数字图像超分辨率重建技术综述

图像超分辨率重建的目的是通过一幅或多幅低分辨率降质图像来估计一幅视觉效果较好的高分辨率图像.它从传统的图像恢复与重建技术而来,利用图像之间的信息互补来获得比单幅图像更多的细节.超分辨率技术主要分为两大类:基于重建的超分辨率技术和基于学习的超分辨率技术.基于重建的超分辨率技术按照特定的退化模型,通过输入的图像来估计高分辨率图像.基于学习的超分辨率技术从训练样本中获取先验知识,对输入图像的信息进行补充,可以获得比基于重建的算法更好的效果.对超分辨技术的算法作了系统的介绍,并指出图像的配准、退化模型的建立、盲估计问题、学习模型的建立、学习算法等仍是图像超分辨率技术中存在的主要问题,也是进一步研究的方向.     

Incorporating non-motion cues into 3D motion segmentation

We address the problem of segmenting an image sequence into rigidly moving 3D objects. An elegant solution to this problem in the case of orthographic projection is the multibody factorization approach in which the measurement matrix is factored into lower rank matrices. Despite progress in factorization algorithms, their performance is still far from satisfactory and in scenes with missing data and noise, most existing algorithms fail.In this paper we propose a method for incorporating 2D non-motion cues (such as spatial coherence) into multibody factorization. We show the similarity of the problem to constrained factor analysis and use the EM algorithm to find the segmentation. We show that adding these cues improves performance in real and synthetic sequences.     

Model-based recognition of 3D objects from single images

In this work, we treat major problems of object recognition which have received relatively little attention lately. Among them are the loss of depth information in the projection from a 3D object to a single 2D image, and the complexity of finding feature correspondences between images. We use geometric invariants to reduce the complexity of these problems. There are no geometric invariants of a projection from 3D to 2D. However, given certain modeling assumptions about the 3D object, such invariants can be found. The modeling assumptions can be either a particular model or a generic assumption about a class of models. Here, we use such assumptions for single-view recognition. We find algebraic relations between the invariants of a 3D model and those of its 2D image under general projective projection. These relations can be described geometrically as invariant models in a 3D invariant space, illuminated by invariant “light rays,” and projected onto an invariant version of the given image. We apply the method to real images     

A statistical learning theory approach for uncertain linear and bilinear matrix inequalities

In this paper, we consider the problem of minimizing a linear functional subject to uncertain linear and bilinear matrix inequalities, which depend in a possibly nonlinear way on a vector of uncertain parameters. Motivated by recent results in statistical learning theory, we show that probabilistic guaranteed solutions can be obtained by means of randomized algorithms. In particular, we show that the Vapnik–Chervonenkis dimension (VC-dimension) of the two problems is finite, and we compute upper bounds on it. In turn, these bounds allow us to derive explicitly the sample complexity of these problems. Using these bounds, in the second part of the paper, we derive a sequential scheme, based on a sequence of optimization and validation steps. The algorithm is on the same lines of recent schemes proposed for similar problems, but improves both in terms of complexity and generality. The effectiveness of this approach is shown using a linear model of a robot manipulator subject to uncertain parameters.     

Matching sets of 3D line segments with application to polygonal arcmatching

In this paper, we consider two sets of corresponding 3D line segments of equal length. We derive a closed-form solution for the coordinate transform (rotation and translation) that gives the best match between the two sets; best in the sense of a least-squares distance measure between the sets. We use these results as the basis to construct efficient algorithms for solving other problems in computer vision. Specifically, we address the problem of matching polygonal arcs, that is, the problem of finding a match between a short arc and a piece of long arc