Mode estimation in high-dimensional spaces with flat-top kernels: Application to image denoising

Mode estimation is extensively studied in statistics. One of the most widely used methods of mode estimation is hill-climbing on a kernel density estimator with gradient ascent or a fixed-point approach. Within this framework, Gaussian kernels proves to be a natural and intuitive option for non-parametric density estimation. This paper shows that in the case of high-dimensional data, mode estimation can be improved by using differently shaped kernels, called flat-top kernels. The improvement are illustrated with an image denoising application, in which pictures are decomposed into small patches, i.e. groups of adjacent pixels, that are vectorized. Noise in the patches can be attenuated by substituting them with the closest mode in the observed distribution of patches. The quality of the denoised picture then depends on the accuracy of mode estimation in a high-dimensional space. Experiments conducted on usual benchmarks in the image processing community show that flat-top kernels outperform the Gaussian one.     

Scale Dependency of Image Derivatives for Feature Measurement in Curvilinear Structures

Extraction of image features is a crucial step in many image analysis tasks. In feature extraction methods Gaussian derivative kernels are frequently utilized. Blurring of the image due to convolution with these kernels gives rise to feature measures different from the intended value in the original image. We propose to solve this problem by explicitly modeling the scale dependency of derivatives combined with measurement of derivatives at multiple scales. This approach is illustrated in methods for feature measurement in curvilinear structures. Results in 3D Confocal Images confirm that modelling of scale behavior of derivatives results in improved methods for center line localization in curved line structures and enables curvature and diameter measurement.     

Robust structure from motion with affine camera via low-rank matrix recovery

We present a novel approach to structure from motion that can deal with missing data and outliers with an affine camera. We model the corruptions as sparse error. Therefore the structure from motion problem is reduced to the problem of recovering a low-rank matrix from corrupted observations. We first decompose the matrix of trajectories of features into low-rank and sparse components by nuclear-norm and l1-norm minimization, and then obtain the motion and structure from the low-rank components by the classical factorization method. Unlike pervious methods, which have some drawbacks such as depending on the initial value selection and being sensitive to the large magnitude errors, our method uses a convex optimization technique that is guaranteed to recover the low-rank matrix from highly corrupted and incomplete observations. Experimental results demonstrate that the proposed approach is more efficient and robust to large-scale outliers.     

Implicitly Weighted Methods in Robust Image Analysis

This paper is devoted to highly robust statistical methods with applications to image analysis. The methods of the paper exploit the idea of implicit weighting, which is inspired by the highly robust least weighted squares regression estimator. We use a correlation coefficient based on implicit weighting of individual pixels as a highly robust similarity measure between two images. The reweighted least weighted squares estimator is considered as an alternative regression estimator with a clear interpretation. We apply implicit weighting to dimension reduction by means of robust principal component analysis. Highly robust methods are exploited in tasks of face localization and face detection in a database of 2D images. In this context we investigate a method for outlier detection and a filter for image denoising based on implicit weighting.     

Multiple Light Source Estimation in a Single Image

Many high‐level image processing tasks require an estimate of the positions, directions and relative intensities of the light sources that illuminated the depicted scene. In image‐based rendering, augmented reality and computer vision, such tasks include matching image contents based on illumination, inserting rendered synthetic objects into a natural image, intrinsic images, shape from shading and image relighting. Yet, accurate and robust illumination estimation, particularly from a single image, is a highly ill‐posed problem. In this paper, we present a new method to estimate the illumination in a single image as a combination of achromatic lights with their 3D directions and relative intensities. In contrast to previous methods, we base our azimuth angle estimation on curve fitting and recursive refinement of the number of light sources. Similarly, we present a novel surface normal approximation using an osculating arc for the estimation of zenith angles. By means of a new data set of ground‐truth data and images, we demonstrate that our approach produces more robust and accurate results, and show its versatility through novel applications such as image compositing and analysis.     

Accurate and fast 3D head pose estimation with noisy RGBD images

Head pose estimation plays an essential role in many high-level face analysis tasks. However, accurate and robust pose estimation with existing approaches remains challenging. In this paper, we propose a novel method for accurate three-dimensional (3D) head pose estimation with noisy depth maps and high-resolution color images that are typically produced by popular RGBD cameras such as the Microsoft Kinect. Our method combines the advantages of the high-resolution RGB image with the 3D information of the depth image. For better accuracy and robustness, features are first detected using only the color image, and then the 3D feature points used for matching are obtained by combining depth information. The outliers are then filtered with depth information using rules proposed for depth consistency, normal consistency, and re-projection consistency, which effectively eliminate the influence of depth noise. The pose parameters are then iteratively optimized using the Extended LM (Levenberg-Marquardt) method. Finally, a Kalman filter is used to smooth the parameters. To evaluate our method, we built a database of more than 10K RGBD images with ground-truth poses recorded using motion capture. Both qualitative and quantitative evaluations show that our method produces notably smaller errors than previous methods.     

Efficient and robust model fitting with unknown noise scale

This paper addresses the general problem of robust parametric model estimation from data that has both an unknown (and possibly majority) fraction of outliers as well as an unknown scale of measurement noise. We focus on computer vision applications from image correspondences, such as camera resectioning, estimation of the fundamental matrix or relative pose for 3D reconstruction, and estimation of 2D homographies for image registration and motion segmentation, although there are many other applications. In practice, these methods typically rely on a predefined inlier thresholds because automatic scale detection is usually too unreliable or too slow. We propose a new method for robust estimation with automatic scale detection that is faster, more precise and more robust than previous alternatives, and show that it can be practically applied to these problems.     

基于前向查找和均值漂移的点模型鲁棒降噪算法

针对点模型提出了基于前向查找和均值漂移两种鲁棒统计方法的滤波算法。前向查找算法根据残差图自动检测离群点，并将输入的点云数据划分为多个不带离群点的最优局部降噪邻域。对局部邻域进行加权协方差分析，估计出该邻域的最小二乘拟合平面。在局部邻域内估计采样点的核密度函数并通过均值漂移算法计算它的局部最大值点，核密度函数的局部最大值点确定了点云数据的聚类中心并能准确逼近采样点曲面，将每一个采样点漂移到密度函数的局部最大值点，使点云曲面收敛为一个稳定的三维数字模型。实验结果表明，本文的算法是鲁棒的，能在有效剔除点模型表面噪声的同时较好地保持模型表面的尖锐特征。     

Determining Image Origin and Integrity Using Sensor Noise

In this paper, we provide a unified framework for identifying the source digital camera from its images and for revealing digitally altered images using photo-response nonuniformity noise (PRNU), which is a unique stochastic fingerprint of imaging sensors. The PRNU is obtained using a maximum-likelihood estimator derived from a simplified model of the sensor output. Both digital forensics tasks are then achieved by detecting the presence of sensor PRNU in specific regions of the image under investigation. The detection is formulated as a hypothesis testing problem. The statistical distribution of the optimal test statistics is obtained using a predictor of the test statistics on small image blocks. The predictor enables more accurate and meaningful estimation of probabilities of false rejection of a correct camera and missed detection of a tampered region. We also include a benchmark implementation of this framework and detailed experimental validation. The robustness of the proposed forensic methods is tested on common image processing, such as JPEG compression, gamma correction, resizing, and denoising.     

医学3D计算机视觉：研究进展和挑战

医学影像的诊断是许多临床决策的基础,而医学影像的智能分析是医疗人工智能的重要组成部分。与此同时,随着越来越多3D空间传感器的兴起和普及,3D计算机视觉正变得越发重要。本文关注医学影像分析和3D计算机的交叉领域,即医学3D计算机视觉或医学3D视觉。本文将医学3D计算机视觉系统划分为任务、数据和表征3个层面,并结合最新文献呈现这3个层面的研究进展。在任务层面,介绍医学3D计算机视觉中的分类、分割、检测、配准和成像重建,以及这些任务在临床诊断和医学影像分析中的作用和特点。在数据层面,简要介绍了医学3D数据中最重要的数据模态：包括计算机断层成像（computed tomography,CT）、磁共振成像（magnetic resonance imaging,MRI）、正电子放射断层成像（positron emission tomography,PET）等,以及一些新兴研究提出的其他数据格式。在此基础上,整理了医学3D计算机视觉中重要的研究数据集,并标注其数据模态和主要视觉任务。在表征层面,介绍并讨论了2D网络、3D网络和混合网络在医学3D数据的表征学习上的优缺点。此外,针对医学影像中普遍存在的小数据问题,重点讨论了医学3D数据表征学习中的预训练问题。最后,总结了目前医学3D计算机视觉的研究现状,并指出目前尚待解决的研究挑战、问题和方向。     

视觉Transformer识别任务研究综述

Transformer模型在自然语言处理领域取得了很好的效果,同时因其能够更好地连接视觉和语言,也激发了计算机视觉界的极大兴趣。本文总结了视觉Transformer处理多种识别任务的百余种代表性方法,并对比分析了不同任务内的模型表现,在此基础上总结了每类任务模型的优点、不足以及面临的挑战。根据识别粒度的不同,分别着眼于诸如图像分类、视频分类的基于全局识别的方法,以及目标检测、视觉分割的基于局部识别的方法。考虑到现有方法在3种具体识别任务的广泛流行,总结了在人脸识别、动作识别和姿态估计中的方法。同时,也总结了可用于多种视觉任务或领域无关的通用方法的研究现状。基于Transformer的模型实现了许多端到端的方法,并不断追求准确率与计算成本的平衡。全局识别任务下的Transformer模型对补丁序列切分和标记特征表示进行了探索,局部识别任务下的Transformer模型因能够更好地捕获全局信息而取得了较好的表现。在人脸识别和动作识别方面,注意力机制减少了特征表示的误差,可以处理丰富多样的特征。Transformer可以解决姿态估计中特征错位的问题,有利于改善基于回归的方法性能,还减少了三维估计时深度映射所产生的歧义。大量探索表明视觉Transformer在识别任务中的有效性,并且在特征表示或网络结构等方面的改进有利于提升性能。     

Line Extraction in 2D Range Images for Mobile Robotics

This paper presents a geometrical feature detection framework for use with conventional 2D laser rangefinders. This framework is composed of three main procedures: data pre-processing, breakpoint detection and line extraction. In data pre-processing, low-level data organization and processing are discussed, with emphasis to sensor bias compensation. Breakpoint detection allows to determine sequences of measurements which are not interrupted by scanning surface changing. Two breakpoint detectors are investigated, one based on adaptive thresholding, and the other on Kalman filtering. Implementation and tuning of both detectors are also investigated. Line extraction is performed to each continuous scan sequence in a range image by applying line kernels. We have investigated two classic kernels, commonly used in mobile robots, and our Split-and-Merge Fuzzy (SMF) line extractor. SMF employs fuzzy clustering in a split-and-merge framework without the need to guess the number of clusters. Qualitative and quantitative comparisons using simulated and real images illustrate the main characteristics of the framework when using different methods for breakpoint and line detection. These comparisons illustrate the characteristics of each estimator, which can be exploited according to the platform computing power and the application accuracy requirements.     

Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks

In this paper we propose a deep learning solution to age estimation from a single face image without the use of facial landmarks and introduce the IMDB-WIKI dataset, the largest public dataset of face images with age and gender labels. If the real age estimation research spans over decades, the study of apparent age estimation or the age as perceived by other humans from a face image is a recent endeavor. We tackle both tasks with our convolutional neural networks (CNNs) of VGG-16 architecture which are pre-trained on ImageNet for image classification. We pose the age estimation problem as a deep classification problem followed by a softmax expected value refinement. The key factors of our solution are: deep learned models from large data, robust face alignment, and expected value formulation for age regression. We validate our methods on standard benchmarks and achieve state-of-the-art results for both real and apparent age estimation.     

Minimum reliable scale selection in 3D

Multiscale analysis is often required in image processing applications because image features are optimally detected at different levels of resolution. With the advance of high-resolution 3D imaging, the extension of multiscale analysis to higher dimensions is necessary. This paper extends an existing 2D scale selection method, known as the minimum reliable scale, to 3D volumetric images. The method is applied to 3D boundary detection and is illustrated in examples from biomedical imaging. The experimental results show that the 3D scale selection improves the detection of edges over single scale operators using as few as three different scales.     

3D遮挡模型引导的光场图像深度获取

目的 光场相机可以通过单次曝光同时从多个视角采样单个场景,在深度估计领域具有独特优势。消除遮挡的影响是光场深度估计的难点之一。现有方法基于2D场景模型检测各视角遮挡状态,但是遮挡取决于所采样场景的3D立体模型,仅利用2D模型无法精确检测,不精确的遮挡检测结果将降低后续深度估计精度。针对这一问题,提出了3D遮挡模型引导的光场图像深度获取方法。方法 向2D模型中的不同物体之间添加前后景关系和深度差信息,得到场景的立体模型,之后在立体模型中根据光线的传输路径推断所有视角的遮挡情况并记录在遮挡图（occlusion map）中。在遮挡图引导下,在遮挡和非遮挡区域分别使用不同成本量进行深度估计。在遮挡区域,通过遮挡图屏蔽被遮挡视角,基于剩余视角的成像一致性计算深度;在非遮挡区域,根据该区域深度连续特性设计了新型离焦网格匹配成本量,相比传统成本量,该成本量能够感知更广范围的色彩纹理,以此估计更平滑的深度图。为了进一步提升深度估计的精度,根据遮挡检测和深度估计的依赖关系设计了基于最大期望（exception maximization,EM）算法的联合优化框架,在该框架下,遮挡图和深度图通过互相引导的方式相继提升彼此精度。结果 实验结果表明,本文方法在大部分实验场景中,对于单遮挡、多遮挡和低对比度遮挡在遮挡检测和深度估计方面均能达到最优结果。均方误差（mean square error,MSE）对比次优结果平均降低约19.75%。结论 针对遮挡场景的深度估计,通过理论分析和实验验证,表明3D遮挡模型相比传统2D遮挡模型在遮挡检测方面具有一定优越性,本文方法更适用于复杂遮挡场景的深度估计。     

Local Upsampling Fourier Transform for accurate 2D/3D image registration

An accurate image registration method based on Local Upsampling Fourier Transform (LUFT) is developed in this paper. It uses a hierarchical strategy to estimate more accurate image pair’s registration parameters, which consists of a coarse estimation and a robust and efficient refinement stage as well. The initial parameter is estimated through a conventional Phase Only Correlation (POC) method in the coarse stage, and then it is refined by the Local Upsampling Fourier Transform in frequency domain to achieve higher accuracy. Furthermore, as will be shown in many experiments, the LUFT can achieve more accurate translation and rotation estimation, and it is efficient, robust to noise, and it can be applied to accurate 2D and 3D image rotation and translation estimation.     

Occlusion-Aware 3D Morphable Models and an Illumination Prior for Face Image Analysis

Faces in natural images are often occluded by a variety of objects. We propose a fully automated, probabilistic and occlusion-aware 3D morphable face model adaptation framework following an analysis-by-synthesis setup. The key idea is to segment the image into regions explained by separate models. Our framework includes a 3D morphable face model, a prototype-based beard model and a simple model for occlusions and background regions. The segmentation and all the model parameters have to be inferred from the single target image. Face model adaptation and segmentation are solved jointly using an expectation–maximization-like procedure. During the E-step, we update the segmentation and in the M-step the face model parameters are updated. For face model adaptation we apply a stochastic sampling strategy based on the Metropolis–Hastings algorithm. For segmentation, we apply loopy belief propagation for inference in a Markov random field. Illumination estimation is critical for occlusion handling. Our combined segmentation and model adaptation needs a proper initialization of the illumination parameters. We propose a RANSAC-based robust illumination estimation technique. By applying this method to a large face image database we obtain a first empirical distribution of real-world illumination conditions. The obtained empirical distribution is made publicly available and can be used as prior in probabilistic frameworks, for regularization or to synthesize data for deep learning methods.     

Histogram-Based Optical Flow for Motion Estimation in Ultrasound Imaging

Motion estimation on ultrasound data is often referred to as ‘Speckle Tracking’ in clinical environments and plays an important role in diagnosis and monitoring of cardiovascular diseases and the identification of abnormal cardiac motion. The impact of physical effects in the process of data acquisition raises many problems for conventional image processing techniques. The most significant difference to other medical data is its high level of speckle noise, which has completely different characteristics from other noise models, e.g., additive Gaussian noise. In this paper we address the problem of multiplicative speckle noise for motion estimation techniques that are based on optical flow methods and prove that the influence of this noise leads to wrong correspondences between image regions if not taken into account. To overcome these problems we propose the use of local statistics and introduce an optical flow method which uses histograms as discrete representations of local statistics for motion analysis. We show that this approach is more robust under the presence of speckle noise than classical optical flow methods.