Robust radiometric calibration and vignetting correction

In many computer vision systems, it is assumed that the image brightness of a point directly reflects the scene radiance of the point. However, the assumption does not hold in most cases due to nonlinear camera response function, exposure changes, and vignetting. The effects of these factors are most visible in image mosaics and textures of 3D models where colors look inconsistent and notable boundaries exist. In this paper, we propose a full radiometric calibration algorithm that includes robust estimation of the radiometric response function, exposures, and vignetting. By decoupling the effect of vignetting from the response function estimation, we approach each process in a manner that is robust to noise and outliers. We verify our algorithm with both synthetic and real data which shows significant improvement compared to existing methods. We apply our estimation results to radiometrically align images for seamless mosaics and 3D model textures. We also use our method to create high dynamic range (HDR) mosaics which are more representative of the scene than normal mosaics.     

Single-Image Vignetting Correction

In this paper, we propose a method for robustly determining the vignetting function given only a single image. Our method is designed to handle both textured and untextured regions in order to maximize the use of available information. To extract vignetting information from an image, we present adaptations of segmentation techniques that locate image regions with reliable data for vignetting estimation. Within each image region, our method capitalizes on the frequency characteristics and physical properties of vignetting to distinguish it from other sources of intensity variation. Rejection of outlier pixels is applied to improve the robustness of vignetting estimation. Comprehensive experiments demonstrate the effectiveness of this technique on a broad range of images with both simulated and natural vignetting effects. Causes of failures using the proposed algorithm are also analyzed.     

Camera Spectral Sensitivity and White Balance Estimation from Sky Images

Photometric camera calibration is often required in physics-based computer vision. There have been a number of studies to estimate camera response functions (gamma function), and vignetting effect from images. However less attention has been paid to camera spectral sensitivities and white balance settings. This is unfortunate, since those two properties significantly affect image colors. Motivated by this, a method to estimate camera spectral sensitivities and white balance setting jointly from images with sky regions is introduced. The basic idea is to use the sky regions to infer the sky spectra. Given sky images as the input and assuming the sun direction with respect to the camera viewing direction can be extracted, the proposed method estimates the turbidity of the sky by fitting the image intensities to a sky model. Subsequently, it calculates the sky spectra from the estimated turbidity. Having the sky \(RGB\) values and their corresponding spectra, the method estimates the camera spectral sensitivities together with the white balance setting. Precomputed basis functions of camera spectral sensitivities are used in the method for robust estimation. The whole method is novel and practical since, unlike existing methods, it uses sky images without additional hardware, assuming the geolocation of the captured sky is known. Experimental results using various real images show the effectiveness of the method.     

Evaluation of Stereo Matching Costs on Images with Radiometric Differences

Stereo correspondence methods rely on matching costs for computing the similarity of image locations. We evaluate the insensitivity of different costs for passive binocular stereo methods with respect to radiometric variations of the input images. We consider both pixel-based and window-based variants like the absolute difference, the sampling-insensitive absolute difference, and normalized cross correlation, as well as their zero-mean versions. We also consider filters like LoG, mean, and bilateral background subtraction (BilSub) and nonparametric measures like Rank, SoftRank, Census, and Ordinal. Finally, hierarchical mutual information (HMI) is considered as pixelwise cost. Using stereo data sets with ground-truth disparities taken under controlled changes of exposure and lighting, we evaluate the costs with a local, a semiglobal, and a global stereo method. We measure the performance of all costs in the presence of simulated and real radiometric differences, including exposure differences, vignetting, varying lighting, and noise. Overall, the ranking of methods across all data sets and experiments appears to be consistent. Among the best costs are BilSub, which performs consistently very well for low radiometric differences; HMI, which is slightly better as pixelwise matching cost in some cases and for strong image noise; and Census, which showed the best and most robust overall performance.     

基于在线光度标定的单目直接视觉SLAM

为了减少场景点的光照变化对直接视觉SLAM （simultaneous localization and mapping,SLAM）的影响,在稀疏直接法（direct sparse odometry with loop closure,LDSO）的基础上对其优化,结合光度标定来增强系统的整体性能。直接视觉SLAM,其图像配对的基本假设是建立在灰度一致性之上的,即同一个场景点在多个图像中是以恒定亮度值出现的。为了更好地满足这一假设,采用KLT （Kanade-Lucas-Tracker）来跟踪关键点,利用带光度参数的像素跟踪模型建立优化方程,优化序列的光度参数（曝光时间、晕影和响应函数）,从而实现实时曝光补偿,增强了系统视觉前端的跟踪稳定性。最后,在公开数据集上进行实验。实验结果标明所提方法在部分序列上能有效地提高系统性能。     

Image-Based Rendering Using Image-Based Priors

Given a set of images acquired from known viewpoints, we describe a method for synthesizing the image which would be seen from a new viewpoint. In contrast to existing techniques, which explicitly reconstruct the 3D geometry of the scene, we transform the problem to the reconstruction of colour rather than depth. This retains the benefits of geometric constraints, but projects out the ambiguities in depth estimation which occur in textureless regions.On the other hand, regularization is still needed in order to generate high-quality images. The papers second contribution is to constrain the generated views to lie in the space of images whose texture statistics are those of the input images. This amounts to an image-based prior on the reconstruction which regularizes the solution, yielding realistic synthetic views. Examples are given of new view generation for cameras interpolated between the acquisition viewpoints—which enables synthetic steadicam stabilization of a sequence with a high level of realism.     

星载SAR影像上船舶方位向模糊去除算法

目的 掌握海上船舶分布状态对于海上交通流分析和通航安全管理具有重要作用。遥感技术,特别是星载合成孔径雷达(SAR)技术的发展,为大范围海上船舶检测提供了有效的手段,但受SAR成像机制影响,海上船舶目标在星载SAR影像上通常存在着不同程度的方位向模糊噪声,这些噪声易被误判为船舶,导致船舶识别中虚警率提高。方法 本文简述了方位向模糊噪声的产生原因,提出了一种新的星载SAR影像上船舶方位向模糊去除算法,该算法的核心是构建目标方位向角度一致性、方位向位置偏移距离和方位向模糊能量衰减3个判别规则,对潜在SAR影像亮斑目标进行逐层筛选,实现船舶真实目标和方位向模糊目标的判别。结果 选取中国渤海海域和黄海海域的30 m分辨率的Radarsat-2数据进行案例分析,并与船舶自动识别系统(AIS)实测数据进行比对校验,结果表明,传统的双参数恒虚警率(CFAR)算法和基于K分布的CFAR等算法对于船舶难以剔除方位向模糊,容易造成虚警,而本文算法对实验影像的船舶方位向模糊去除准确率优于95.8%,能够有效剔除船舶方位向模糊。结论 该算法为星载SAR影像上船舶方位向模糊去除提供了新的手段,有助于提高SAR影像上船舶目标检测的准确性。     

低通滤波下约束对数强度熵的图像渐晕校正

渐晕是图像中像素光强沿径向方向衰减的现象，为了解决其对计算机视觉任务和图像处理精度的影响，提出了低通滤波下约束对数强度熵的图像渐晕校正方法。首先，使用偶数项的六阶多项式函数建立渐晕模型；其次，通过低通滤波计算最小的目标图像对数强度熵，在目标值的约束下求出既满足渐晕函数变化规律又能减小图像的对数强度熵的渐晕模型的最优参数解；最后，对渐晕图像采用渐晕模型的逆向补偿校正渐晕。采用结构相似性指标（SSIM）和均方根误差(RMSE)度量渐晕校正效果，结果表明，所提方法不仅能有效恢复渐晕区域的亮度信息，得到真实、自然的无渐晕图像，而且能有效校正不同程度的渐晕，渐晕校正结果视觉一致性良好。     

Generalized Mosaicing: High Dynamic Range in a Wide Field of View

We present an approach that significantly enhances the capabilities of traditional image mosaicking. The key observation is that as a camera moves, it senses each scene point multiple times. We rigidly attach to the camera an optical filter with spatially varying properties, so that multiple measurements are obtained for each scene point under different optical settings. Fusing the data captured in the multiple images yields an image mosaic that includes additional information about the scene. We refer to this approach as generalized mosaicing. In this paper we show that this approach can significantly extend the optical dynamic range of any given imaging system by exploiting vignetting effects. We derive the optimal vignetting configuration and implement it using an external filter with spatially varying transmittance. We also derive efficient scene sampling conditions as well as ways to self calibrate the vignetting effects. Maximum likelihood is used for image registration and fusion. In an experiment we mounted such a filter on a standard 8-bit video camera, to obtain an image panorama with dynamic range comparable to imaging with a 16-bit camera.     

手机文字识别中的暗角修正技术

为了提高手机图像的识别率,从手机文字扫描识别应用问题出发,在分析手机普遍存在的暗角退化特征的基础上,提出了一种从含有文字信息的图像中提取图像的暗角退化模型的方法;然后根据该暗角模型对输入图像进行复原,最终达到修正手机摄像模组暗角退化的目的。     

张量分析在视差图生成中的应用

针对当前视差图生成算法中常用的匹配代价对光晕等复杂光学畸变比较敏感的问题,在对相机成像模型进行分析的基础上提出了一种基于张量分析的匹配代价。和目前常用的匹配代价相比,该匹配代价不但对光晕效应具有更强的鲁棒性而且能够更加准确的反映图像的局部结构信息,有效的降低了匹配的歧义性。在本文中,张量在积分图像上构造,张量间的距离在黎曼流形上测量;此外,提出了一种简单有效的视差图后续处理方法进一步修正了初始视差图中的不可信视差。在标准测试集和人工图像上的实验结果验证了该方法的有效性。     

True Color Correction of Autonomous Underwater Vehicle Imagery

This paper presents an automated approach to recovering the true color of objects on the seafloor in images collected from multiple perspectives by an autonomous underwater vehicle (AUV) during the construction of three‐dimensional (3D) seafloor models and image mosaics. When capturing images underwater, the water column induces several effects on light that are typically negligible in air, such as color‐dependent attenuation and backscatter. AUVs must typically carry artificial lighting when operating at depths below 20‐30 m; the lighting pattern generated is usually not spatially consistent. These effects cause problems for human interpretation of images, limit the ability of using color to identify benthic biota or quantify changes over multiple dives, and confound computer‐based techniques for clustering and classification. Our approach exploits the 3D structure of the scene generated using structure‐from‐motion and photogrammetry techniques to provide basic spatial data to an underwater image formation model. Parameters that are dependent on the properties of the water column are estimated from the image data itself, rather than using fixed in situ infrastructure, such as reflectance panels or detailed data on water constitutes. The model accounts for distance‐based attenuation and backscatter, camera vignetting and the artificial lighting pattern, recovering measurements of the true color (reflectance) and thus allows us to approximate the appearance of the scene as if imaged in air and illuminated from above. Our method is validated against known color targets using imagery collected in different underwater environments by two AUVs that are routinely used as part of a benthic habitat monitoring program.     

Generalized Rough Sets, Entropy, and Image Ambiguity Measures

Quantifying ambiguities in images using fuzzy set theory has been of utmost interest to researchers in the field of image processing. In this paper, we present the use of rough set theory and its certain generalizations for quantifying ambiguities in images and compare it to the use of fuzzy set theory. We propose classes of entropy measures based on rough set theory and its certain generalizations, and perform rigorous theoretical analysis to provide some properties which they satisfy. Grayness and spatial ambiguities in images are then quantified using the proposed entropy measures. We demonstrate the utility and effectiveness of the proposed entropy measures by considering some elementary image processing applications. We also propose a new measure called average image ambiguity in this context.     

Transfer of content-aware vignetting effect from paintings to photographs

This paper discusses how the vignetting effect of paintings may be transferred to photographs, with attention to center-corner contrast. First, the lightness distribution of both are analyzed. The results show that the painter’s vignette is more complex than that achieved using common digital post-processing methods. It is shown to involve both the 2D and 3D geometry of the scene. Then, an algorithm is developed to transfer the vignetting effect from an example painting to a photograph. The example painting is selected as that has similar contextual geometry with the photograph. The lightness weighting pattern extracted from the selected example painting is adaptively blended with the input photograph to create vignetting effect. In order to avoid over-brightened or over-darkened regions in the enhancement result, the extracted lightness weighting pattern is corrected using a nonlinear curve. A content-aware interpolation method is also proposed to warp the lightness weighting to fit the contextual structure of the photograph. Finally, the local contrast is restored. Experiments show that the proposed algorithm can successfully perform this function. The resulting vignetting effect is more naturally presented with regard to esthetic composition as compared with vignetting achieved with popular software tools and camera models.

     

Generalised residual images’ effect on illumination artifact removal for correspondence algorithms

The intensity (grey value) consistency of image pixels in a sequence or stereo camera setup is of central importance to numerous computer vision applications. Most stereo matching and optical flow algorithms minimise an energy function composed of a data term and a regularity or smoothing term. To date, well performing methods rely on the intensity consistency of the image pixel values to model the data term. Such a simple model fails if the illumination is (even slightly) different between the input images. Amongst other situations, this may happen due to background illumination change over the sequence, different reflectivity of a surface, vignetting, or shading effects.In this paper, we investigate the removal of illumination artifacts and show that generalised residual images substantially improve the accuracy of correspondence algorithms. In particular, we motivate the concept of residual images and show two evaluation approaches using either ground truth correspondence fields (for stereo matching and optical flow algorithms) or errors based on a predicted view (for stereo matching algorithms).     

虚拟摄影机在动画设计中的拟真应用

三维设计软件中的摄影机概念是"完美"的。真实的摄影机带有诸多"缺陷",例如:渐晕、畸变、色散等,为了制造出更加富有真实感的动画,三维设计者需要依照真实摄影机的"缺陷"来调整三维软件中的摄影机。     

组合式建筑立面全景图像的渐晕校正

由于组合式建筑立面图像进行全景拼接时,交界处光照度降低,产生渐晕现象,提出一种基于曲面拟合的渐晕校正方法,对组合式建筑立面全景图像渐晕校正。通过推算角点,分析图像坐标系及世界坐标系之间的对照关联,获取世界坐标系中角点的单位坐标数据,构建相机成像的组合式建筑三维交互模型;采用ORB特征匹配算法实施图像匹配,通过分解单应性矩阵,推算相机内部参数的投影转换矩阵、旋转矩阵和平移矩阵,运用柱面坐标投影方法完成建筑立面图像全景拼接;使用基于二维曲面拟合图像灰度校正方法,计算Hesse矩阵的相关系数,利用Hesse矩阵逐次迭代收敛,获得二维曲面参数值,消除图像渐晕现象。仿真结果证明,所提方法能够有效校正组合式建筑立面全景图像渐晕,且校正后的图像灰度分布均匀,缩短了图像渐晕校正时间。     

The Multi-strand Graph for a PTZ Tracker

High-resolution images can be used to resolve matching ambiguities between trajectory fragments (tracklets), which is a key challenge in multiple-target tracking. A pan–tilt–zoom (PTZ) camera, which can pan, tilt and zoom, is a powerful and efficient tool that offers both close-up views and wide area coverage on demand. The wide area enables tracking of many targets, while the close-up view allows individuals to be identified from high-resolution images of their faces. A central component of a PTZ tracking system is a scheduling algorithm that determines which target to zoom in on, particularly when the high-resolution images are also used for tracklet matching. In this paper, we study this scheduling problem from a theoretical perspective. We propose a novel data structure, the Multi-strand Tracking Graph (MSG), which represents the set of tracklets computed by a tracker and the possible associations between them. The MSG allows efficient scheduling as well as resolving of matching ambiguities between tracklets. The main feature of the MSG is the auxiliary data saved in each vertex, which allows efficient computation while avoiding time-consuming graph traversal. Synthetic data simulations are used to evaluate our scheduling algorithm and to demonstrate its superiority over a naïve one.     

Automatic feature matching in uncalibrated stereo vision through the use of color

One difficult problem in stereo vision is how to establish correspondence between features extracted from a pair of images. The difficulty is due to ambiguities or inconsistencies of available information on images. In this paper, we invetigate stereo correspondence problem in the framework of color stereo vision. We propose the use of a matching consistence (MC) constraint in RGB color space and the generalized epipolar geometry to develop an automatic feature matching algorithm.     

Removal of azimuth ambiguities and detection of a ship: using polarimetric airborne C-band SAR images

Synthetic aperture radar (SAR) imagery from the sea can contain ships and their ambiguities. The ambiguities are visually identifiable due to their high intensities in the low radar backscatter background of sea environments and can be mistaken as ships, resulting in false alarms in ship detection. Analysing polarimetric characteristics of ships and ambiguities, we found that (a) backscattering from a ship consisted of a mixture of single-bounced, double-bounced and depolarized or diffused scattering types due to its complex physical structure; (b) that only a strong single- or double-bounce scatterer produced ambiguities in azimuth that look like relatively strong double- or single-bounce scatterers, respectively; and (c) that eigenvalues corresponding to the single- or double-bounce scattering mechanisms of the ambiguities were high but the eigenvalue corresponding to the depolarized scattering mechanisms of the ambiguities was low. With these findings, we proposed a ship detection method that applies the eigenvalue to differentiate the ship target and azimuth ambiguities. One set of C-band JPL AIRSAR (Jet Propulsion Laboratory Airborne Synthetic Aperture Radar) polarimetric data from the sea have been chosen to evaluate the method that can effectively delineate ships from their azimuth ambiguities.