Scale Selection Properties of Generalized Scale-Space Interest Point Detectors

Scale-invariant interest points have found several highly successful applications in computer vision, in particular for image-based matching and recognition. This paper presents a theoretical analysis of the scale selection properties of a generalized framework for detecting interest points from scale-space features presented in Lindeberg (Int. J. Comput. Vis. 2010, under revision) and comprising:

an enriched set of differential interest operators at a fixed scale including the Laplacian operator, the determinant of the Hessian, the new Hessian feature strength measures I and II and the rescaled level curve curvature operator, as well as

an enriched set of scale selection mechanisms including scale selection based on local extrema over scale, complementary post-smoothing after the computation of non-linear differential invariants and scale selection based on weighted averaging of scale values along feature trajectories over scale.

It is shown how the selected scales of different linear and non-linear interest point detectors can be analyzed for Gaussian blob models. Specifically it is shown that for a rotationally symmetric Gaussian blob model, the scale estimates obtained by weighted scale selection will be similar to the scale estimates obtained from local extrema over scale of scale normalized derivatives for each one of the pure second-order operators. In this respect, no scale compensation is needed between the two types of scale selection approaches. When using post-smoothing, the scale estimates may, however, be different between different types of interest point operators, and it is shown how relative calibration factors can be derived to enable comparable scale estimates for each purely second-order operator and for different amounts of self-similar post-smoothing. A theoretical analysis of the sensitivity to affine image deformations is presented, and it is shown that the scale estimates obtained from the determinant of the Hessian operator are affine covariant for an anisotropic Gaussian blob model. Among the other purely second-order operators, the Hessian feature strength measure I has the lowest sensitivity to non-uniform scaling transformations, followed by the Laplacian operator and the Hessian feature strength measure II. The predictions from this theoretical analysis agree with experimental results of the repeatability properties of the different interest point detectors under affine and perspective transformations of real image data. A number of less complete results are derived for the level curve curvature operator.     

Temporal Scale Selection in Time-Causal Scale Space

When designing and developing scale selection mechanisms for generating hypotheses about characteristic scales in signals, it is essential that the selected scale levels reflect the extent of the underlying structures in the signal. This paper presents a theory and in-depth theoretical analysis about the scale selection properties of methods for automatically selecting local temporal scales in time-dependent signals based on local extrema over temporal scales of scale-normalized temporal derivative responses. Specifically, this paper develops a novel theoretical framework for performing such temporal scale selection over a time-causal and time-recursive temporal domain as is necessary when processing continuous video or audio streams in real time or when modelling biological perception. For a recently developed time-causal and time-recursive scale-space concept defined by convolution with a scale-invariant limit kernel, we show that it is possible to transfer a large number of the desirable scale selection properties that hold for the Gaussian scale-space concept over a non-causal temporal domain to this temporal scale-space concept over a truly time-causal domain. Specifically, we show that for this temporal scale-space concept, it is possible to achieve true temporal scale invariance although the temporal scale levels have to be discrete, which is a novel theoretical construction. The analysis starts from a detailed comparison of different temporal scale-space concepts and their relative advantages and disadvantages, leading the focus to a class of recently extended time-causal and time-recursive temporal scale-space concepts based on first-order integrators or equivalently truncated exponential kernels coupled in cascade. Specifically, by the discrete nature of the temporal scale levels in this class of time-causal scale-space concepts, we study two special cases of distributing the intermediate temporal scale levels, by using either a uniform distribution in terms of the variance of the composed temporal scale-space kernel or a logarithmic distribution. In the case of a uniform distribution of the temporal scale levels, we show that scale selection based on local extrema of scale-normalized derivatives over temporal scales makes it possible to estimate the temporal duration of sparse local features defined in terms of temporal extrema of first- or second-order temporal derivative responses. For dense features modelled as a sine wave, the lack of temporal scale invariance does, however, constitute a major limitation for handling dense temporal structures of different temporal duration in a uniform manner. In the case of a logarithmic distribution of the temporal scale levels, specifically taken to the limit of a time-causal limit kernel with an infinitely dense distribution of the temporal scale levels towards zero temporal scale, we show that it is possible to achieve true temporal scale invariance to handle dense features modelled as a sine wave in a uniform manner over different temporal durations of the temporal structures as well to achieve more general temporal scale invariance for any signal over any temporal scaling transformation with a scaling factor that is an integer power of the distribution parameter of the time-causal limit kernel. It is shown how these temporal scale selection properties developed for a pure temporal domain carry over to feature detectors defined over time-causal spatio-temporal and spectro-temporal domains.     

基于深度信息与尺度空间的仿射不变特征检测*

介绍了一种利用深度信息的仿射区域检测器。这种方法在视角变换的情况下能自动检测出图景中同一物理区域,为后续的识别算法提供了坚实的特征检测基础,在计算机视觉领域有广阔的应用前景。该方法是基于尺度空间理论,这个理论已经在自动尺度选择中有较成熟的应用。提出了利用深度信息估计出3D物体模型的算法,并生成相应的仿射不变的高斯尺度空间,并给出从3D到2D的投射变换的高精度估计方法,以补偿投射变换造成的扭曲形变。因此对特征检测的可靠性将有明显的提高。为了评估本算法的鲁棒性,进行了不同视角的真实图片与合成图片的实验,并与其     

Time-Causal and Time-Recursive Spatio-Temporal Receptive Fields

We present an improved model and theory for time-causal and time-recursive spatio-temporal receptive fields, obtained by a combination of Gaussian receptive fields over the spatial domain and first-order integrators or equivalently truncated exponential filters coupled in cascade over the temporal domain. Compared to previous spatio-temporal scale-space formulations in terms of non-enhancement of local extrema or scale invariance, these receptive fields are based on different scale-space axiomatics over time by ensuring non-creation of new local extrema or zero-crossings with increasing temporal scale. Specifically, extensions are presented about (i) parameterizing the intermediate temporal scale levels, (ii) analysing the resulting temporal dynamics, (iii) transferring the theory to a discrete implementation in terms of recursive filters over time, (iv) computing scale-normalized spatio-temporal derivative expressions for spatio-temporal feature detection and (v) computational modelling of receptive fields in the lateral geniculate nucleus (LGN) and the primary visual cortex (V1) in biological vision. We show that by distributing the intermediate temporal scale levels according to a logarithmic distribution, we obtain a new family of temporal scale-space kernels with better temporal characteristics compared to a more traditional approach of using a uniform distribution of the intermediate temporal scale levels. Specifically, the new family of time-causal kernels has much faster temporal response properties (shorter temporal delays) compared to the kernels obtained from a uniform distribution. When increasing the number of temporal scale levels, the temporal scale-space kernels in the new family do also converge very rapidly to a limit kernel possessing true self-similar scale-invariant properties over temporal scales. Thereby, the new representation allows for true scale invariance over variations in the temporal scale, although the underlying temporal scale-space representation is based on a discretized temporal scale parameter. We show how scale-normalized temporal derivatives can be defined for these time-causal scale-space kernels and how the composed theory can be used for computing basic types of scale-normalized spatio-temporal derivative expressions in a computationally efficient manner.     

一种基于Gabor小波的局部特征尺度提取方法

图像的局部特征尺度在进行特征提取和构造尺度不变量时非常重要。提出了一种基于Gabor小波的局部特征尺度提取方法,该方法利用视皮层简单细胞的2维Gabor函数模型, 构造了一个Gabor尺度空间核函数,利用该核函数计算图像的Gabor尺度空间分解,并在尺度空间中搜索局部极大值作为特征点的固有尺度。实验结果表明,该方法可在不同对比度 条件下有效地提取各类特征的局部尺度,并且相比高斯拉普拉斯(LoG)方法有更好的适应性和可靠性。     

Local feature extraction for iris recognition with automatic scale selection

This paper presents an iris recognition system using automatic scale selection algorithm for iris feature extraction. The proposed system first filters the given iris image adopting a bank of Laplacian of Gaussian (LoG) filters with many different scales and computes the normalized response of every filter. The parameter γ used to normalize the filter responses, is derived by analyzing the scale-space maxima of the blob feature detector responses. Then the maxima normalized response over scales for each point are selected together as the optimal filter outputs of the given iris image and the binary codes for iris feature representation are achieved by encoding these optimal outputs through a zero threshold. Comparison experiment results clearly demonstrate an efficient performance of the proposed algorithm.     

Adaptive determination of filter scales for edge detection

The authors suggest a regularization method for determining scales for edge detection adaptively for each site in the image plane. Specifically, they extend the optimal filter concept of T. Poggio et al. (1984) and the scale-space concept of A. Witkin (1983) to an adaptive scale parameter. To avoid an ill-posed feature synthesis problem, the scheme automatically finds optimal scales adaptively for each pixel before detecting final edge maps. The authors introduce an energy function defined as a functional over continuous scale space. Natural constraints for edge detection are incorporated into the energy function. To obtain a set of optimal scales that can minimize the energy function, a parallel relaxation algorithm is introduced. Experiments for synthetic and natural scenes show the advantages of the algorithm. In particular, it is shown that this system can detect both step and diffuse edges while drastically filtering out the random noise     

The Gaussian scale-space paradigm and the multiscale local jet

A representation of local image structure is proposed which takes into account both the image's spatial structure at a given location, as well as its deep structure, that is, its local behaviour as a function of scale or resolution (scale-space). This is of interest for several low-level image tasks. The proposed basis of scale-space, for example, enables a precise local study of interactions of neighbouring image intensities in the course of the blurring process. It also provides an extrapolation scheme for local image data, obtained at a given spatial location and resolution, to a finite scale-space neighbourhood. This is especially useful for the determination of sampling rates and for interpolation algorithms in a multilocal context. Another, particularly straightforward application is image enhancement or deblurring, which is an instance of data extrapolation in the high-resolution direction.A potentially interesting feature of the proposed local image parametrisation is that it captures a trade-off between spatial and scale extrapolations from a given interior point that do not exceed a given tolerance. This (rade-off suggests the possibility of a fairly coarse scale sampling at the expense of a dense spatial sampling large relative spatial overlap of scale-space kernels).The central concept developed in this paper is an equivalence class called the multiscale local jet, which is a hierarchical, local characterisation of the image in a full scale-space neighbourhood. For this local jet, a basis of fundamental polynomials is constructed that captures the scale-space paradigm at the local level up to any given order.     

A Subdivision Scheme for Continuous-Scale B-Splines and Affine-Invariant Progressive Smoothing

Multiscale representations and progressive smoothing constitutean important topic in different fields as computer vision, CAGD,and image processing. In this work, a multiscale representationof planar shapes is first described. The approach is based oncomputing classical B-splines of increasing orders, andtherefore is automatically affine invariant. The resultingrepresentation satisfies basic scale-space properties at least ina qualitative form, and is simple to implement.The representation obtained in this way is discrete in scale,since classical B-splines are functions in , where k isan integer bigger or equal than two. We present a subdivisionscheme for the computation of B-splines of finite support atcontinuous scales. With this scheme, B-splines representationsin are obtained for any real r in [0, ), andthe multiscale representation is extended to continuous scale.The proposed progressive smoothing receives a discrete set ofpoints as initial shape, while the smoothed curves arerepresented by continuous (analytical) functions, allowing astraightforward computation of geometric characteristics of theshape.     

Linear scale-space

The formulation of afront-end orearly vision system is addressed, and its connection with scale-space is shown. A front-end vision system is designed to establish a convenient format of some sampled scalar field, which is suited for postprocessing by various dedicated routines. The emphasis is on the motivations and implications of symmetries of the environment; they pose natural, a priori constraints on the design of a front-end.The focus is on static images, defined on a multidimensional spatial domain, for which it is assumed that there are no a priori preferred points, directions, or scales. In addition, the front-end is required to be linear. These requirements are independent of any particular image geometry and express the front-end's pure syntactical, bottom up nature.It is shown that these symmetries suffice to establish the functionality properties of a front-end. For each location in the visual field and each inner scale it comprises a hierarchical family of tensorial apertures, known as the Gaussian family, the lowest order of which is the normalised Gaussian. The family can be truncated at any given order in a consistent way. The resulting set constitutes a basis for alocal jet bundle. Note that scale-space theory shows up here without any call upon the prohibition of spurious detail, which, in some way or another, usually forms the basic starting point for diffusion-like scale-space theories.     

Range Image Segmentation through Pattern Analysis of the Multiscale Wavelet Transform

This work presents an image segmentation method for range data that uses multiscale wavelet analysis in combination with statistical pattern recognition. A feature-detection framework based on multiscale analysis and pattern recognition has several potential advantages over other feature detection systems. These advantages are detection of features at different scales (i.e., features of all sizes), robustness, and few or no free parameters. Our system creates a fuzzy edge map and derives a segmentation from this edge detection. A scale-space signature is the vector of measurements at different scales taken at a single point in an image. We analyze these 1-D signatures with traditional pattern-recognition methods. We train a pattern-recognition system with scale-space signatures from the edge points of a training image. Once trained, the system determines the degree ofedgenessof points in a new image. The goal is to create a system that exploits the advantages of a multiscale, pattern-recognition framework.     

Tensor scale: An analytic approach with efficient computation and applications

Scale is a widely used notion in computer vision and image understanding that evolved in the form of scale-space theory where the key idea is to represent and analyze an image at various resolutions. Recently, we introduced a notion of local morphometric scale referred to as “tensor scale” using an ellipsoidal model that yields a unified representation of structure size, orientation and anisotropy. In the previous work, tensor scale was described using a 2-D algorithmic approach and a precise analytic definition was missing. Also, the application of tensor scale in 3-D using the previous framework is not practical due to high computational complexity. In this paper, an analytic definition of tensor scale is formulated for n-dimensional (n-D) images that captures local structure size, orientation and anisotropy. Also, an efficient computational solution in 2- and 3-D using several novel differential geometric approaches is presented and the accuracy of results is experimentally examined. Also, a matrix representation of tensor scale is derived facilitating several operations including tensor field smoothing to capture larger contextual knowledge. Finally, the applications of tensor scale in image filtering and n-linear interpolation are presented and the performance of their results is examined in comparison with respective state-of-art methods. Specifically, the performance of tensor scale based image filtering is compared with gradient and Weickert’s structure tensor based diffusive filtering algorithms. Also, the performance of tensor scale based n-linear interpolation is evaluated in comparison with standard n-linear and windowed-sinc interpolation methods.     

Analysis of aerosol images using the scale-space primal sketch

We outline a method to analyze aerosol images using the scale-space representation. The pictures, which are photographs of an aerosol generated by a fuel injector, contain phenomena that by a human observer are perceived as periodic or oscillatory structures. The presence of these structures is not immediately apparent since the periodicity manifests itself at a coarse level of scale while the dominating objects in the images are small dark blobs, that is, fine scale objects. Experimentally, we illustrate that the scale-space theory provides an objective method to bring out these events. However, in this form the method still relies on a subjective observer in order to extract and verify the existence of the periodic phenomena.Then we extend the analysis by adding a recently developed image analysis concept called the scale-space primal sketch. With this tool, we are able to extract significant structures from a grey-level image automatically without any strong a priori assumptions about either the shape or the scale (size) of the primitives. Experiments demonstrate that the periodic drop clusters we perceived in the image are detected by the algorithm as significant image structures. These results provide objective evidence verifying the existence of oscillatory phenomena.     

基于Gabor能量空间的尺度不变兴趣点检测方法

兴趣点检测是中层视觉感知过程的关键步骤,也是众多机器视觉系统的重要组成部分.此前的大多数兴趣点检测子都是针对特殊的二维图像结构设计的,比如角点、交叉点、端点等,所以对与其差别较大的特征不能检测.采用在Gabor能量空间中迭代搜索的方法,本文提出了一种尺度不变兴趣点检测子.基于结构不同的二维图像特征在相频域中表现相似的特点,该检测子能检测大多数特征.首先,基于Gabor滤波器响应获得一系列能量图像,通过极值点检测得到候选兴趣点;其次,使用一种迭代方法同时选择特征尺度与精确定位特征点位置;最后为了提高算法的实时性,采用了一种递推方法加速能量图像的计算过程.实验结果表明相对于其它检测子,本文提出的方法具有更广泛的适应性,并且在旋转、尺度、光照等变化下具有良好的稳定性.     

基于Gabor能量空间的尺度不变兴趣点检测方法(英文)

Interest point detection is a fundamental issue in many intermediate level vision problems and plays a significant role in vision systems. The previous interest point detectors are designed to detect some special image structures such as corners,junctions, line terminations and so on. These detectors based on some simplified 2D feature models will not work for image features that differ significantly from the models. In this paper, a scale invariant interest point detector, which is appropriate for most types of image features, is proposed based on an iterative method in the Gabor based energy space. It detects interest points by noting that there are some similarities in the phase domain for all types of image features, which are obtained by different detectors respectively. Firstly, this approach obtains the positions of candidate points by detecting the local maxima of a series of energy maps constructed by Gabor filter responses.Secondly, an iterative algorithm is adopted to select the corresponding characteristic scales and accurately locate the interest points simultaneously in the Gabor based energy space. Finally,in order to improve the real-time performance of the approach, a fast implementation of Gabor function is used to accelerate the process of energy space construction. Experiments show that this approach has a broader applicability than the other detectors and has a good performance under rotation and some other image changes.