The Monogenic Scale-Space: A Unifying Approach to Phase-Based Image Processing in Scale-Space

In this paper we address the topics of scale-space and phase-based image processing in a unifying framework. In contrast to the common opinion, the Gaussian kernel is not the unique choice for a linear scale-space. Instead, we chose the Poisson kernel since it is closely related to the monogenic signal, a 2D generalization of the analytic signal, where the Riesz transform replaces the Hilbert transform. The Riesz transform itself yields the flux of the Poisson scale-space and the combination of flux and scale-space, the monogenic scale-space, provides the local features phase-vector and attenuation in scale-space. Under certain assumptions, the latter two again form a monogenic scale-space which gives deeper insight to low-level image processing. In particular, we discuss edge detection by a new approach to phase congruency and its relation to amplitude based methods, reconstruction from local amplitude and local phase, and the evaluation of the local frequency.     

Linear Scale-Space has First been Proposed in Japan

Linear scale-space is considered to be a modern bottom-up tool in computer vision. The American and European vision community, however, is unaware of the fact that it has already been axiomatically derived in 1959 in a Japanese paper by Taizo Iijima. This result formed the starting point of vast linear scale-space research in Japan ranging from various axiomatic derivations over deep structure analysis to applications to optical character recognition. Since the outcomes of these activities are unknown to western scale-space researchers, we give an overview of the contribution to the development of linear scale-space theories and analyses. In particular, we review four Japanese axiomatic approaches that substantiate linear scale-space theories proposed between 1959 and 1981. By juxtaposing them to ten American or European axiomatics, we present an overview of the state-of-the-art in Gaussian scale-space axiomatics. Furthermore, we show that many techniques for analysing linear scale-space have also been pioneered by Japanese researchers.     

Linear Scale-Space Theory from Physical Principles

In the past decades linear scale-space theory was derived on the basis of various axiomatics. In this paper we revisit these axioms and show that they merely coincide with the following physical principles, namely that the image domain is a Galilean space, that the total energy exchange between a region and its surrounding is preserved under linear filtering and that the physical observables should be invariant under the group of similarity transformations. These observables are elements of the similarity jet spanned by natural coordinates and differential energies read out by a vision system.Furthermore, linear scale-space theory is extended to spatio-temporal images on bounded and curved domains. Our theory permits a delay-operation at the present moment which is in agreement with the motion detection model of Reichardt. In this respect our theory deviates from that of Koenderink which requires additional syntactical operators to realise such a delay-operation.Finally, the semi-discrete and discrete linear scale-space theories are derived by discretising the continuous theories following the theory of stochastic processes. The relation and difference between our stochastic approach and that of Lindeberg is pointed out. The connection between continuous and (semi-)discrete sale-space theory for infinitely high scales observed by Lindeberg is refined by applying appropriate scaling limits. It is shown that Lindeberg's requirement of normalisation for one-dimensional discrete Green's functions can be incorporated into our theory for arbitrary dimensional discrete Green's functions, parameter determination can be avoided, and the requirement of operation at even and odd coordinates sum can be guaranteed simultaneously by taking a normalised linear combination of the identity operator and the first step discrete Green's functions. The new discrete Green's functions are still intimately related to the continuous Green's functions and appear to coincide with pyramidal discrete Green's functions.     

Pseudo-Linear Scale-Space Theory

It has been observed that linear, Gaussian scale-space, and nonlinear, morphological erosion and dilation scale-spaces generated by a quadratic structuring function have a lot in common. Indeed, far-reaching analogies have been reported, which seems to suggest the existence of an underlying isomorphism. However, an actual mapping appears to be missing.In the present work a one-parameter isomorphism is constructed in closed-form, which encompasses linear and both types of morphological scale-spaces as (non-uniform) limiting cases. The unfolding of the one-parameter family provides a means to transfer known results from one domain to the other. Moreover, for any fixed and non-degenerate parameter value one obtains a novel type of pseudo-linear multiscale representation that is, in a precise way, in-between the familiar ones. This is of interest in its own right, as it enables one to balance pros and cons of linear versus morphological scale-space representations in any particular situation.     

Scale-Space Behavior of Planar-Curve Corners

The curvature scale-space (CSS) technique is suitable for extracting curvature features from objects with noisy boundaries. To detect corner points in a multiscale framework, Rattarangsi and Chin investigated the scale-space behavior of planar-curve corners. Unfortunately, their investigation was based on an incorrect assumption, viz., that planar curves have no shrinkage under evolution. In the present paper, this mistake is corrected. First, it is demonstrated that a planar curve may shrink nonuniformly as it evolves across increasing scales. Then, by taking into account the shrinkage effect of evolved curves, the CSS trajectory maps of various corner models are investigated and their properties are summarized. The scale-space trajectory of a corner may either persist, vanish, merge with a neighboring trajectory, or split into several trajectories. The scale-space trajectories of adjacent corners may attract each other when the corners have the same concavity, or repel each other when the corners have opposite concavities. Finally, we present a standard curvature measure for computing the CSS maps of digital curves, with which it is shown that planar-curve corners have consistent scale-space behavior in the digital case as in the continuous case.     

Regularization, Scale-Space, and Edge Detection Filters

Computational vision often needs to deal with derivatives ofdigital images. Such derivatives are not intrinsic properties ofdigital data; a paradigm is required to make them well-defined.Normally, a linear filtering is applied. This can be formulated interms of scale-space, functional minimization, or edge detectionfilters. The main emphasis of this paper is to connect these theoriesin order to gain insight in their similarities and differences. We donot want, in this paper, to take part in any discussion of how edgedetection must be performed, but will only link some of the current theories. We take regularization (or functional minimization) as astarting point, and show that it boils down to Gaussian scale-space ifwe require scale invariance and a semi-group constraint to besatisfied. This regularization implies the minimization of afunctional containing terms up to infinite order of differentiation.If the functional is truncated at second order, the Canny-Deriche filter arises. It is also shown that higher dimensional regularizationboils down to a rotated version of the one dimensional case, whenCartesian invariance is imposed and the image is vanishing at theborders. This means that the results from 1D regularization can beeasily generalized to higher dimensions. Finally we show how anefficient implementation of regularization of order n can be made byrecursive filtering using 2n multiplications and additions peroutput element without introducing any approximation.     

基于视觉特征信息量度量的高斯尺度参数自适应算法

为了避免计算过于复杂或因丢弃过多关键信息而造成失真过大的问题,在高斯尺度空间的构造中应正确选用尺度参数,以使图像信息的变化呈现均匀的特点。目前,许多高斯尺度空间应用中采用的层之间的尺度参数关系并不明确,使得分层效果不理想。本文基于视觉特征模型,提出一种自适应高斯尺度参数的算法,并通过对SAR图像降噪处理对比试验验证了它的有效性,从而为图像的高层次处理如目标识别等提供了信息量稳定变化的尺度空间。     

An Analysis of the Factors Affecting Keypoint Stability in Scale-Space

The most popular image matching algorithm SIFT, introduced by D. Lowe a decade ago, has proven to be sufficiently scale invariant to be used in numerous applications. In practice, however, scale invariance may be weakened by various sources of error inherent to the SIFT implementation affecting the stability and accuracy of keypoint detection. The density of the sampling of the Gaussian scale-space and the level of blur in the input image are two of these sources. This article presents a numerical analysis of their impact on the extracted keypoints stability. Such an analysis has both methodological and practical implications, on how to compare feature detectors and on how to improve SIFT. We show that even with a significantly oversampled scale-space numerical errors prevent from achieving perfect stability. Usual strategies to filter out unstable detections (e.g., poorly contrasted extrema) are shown to be inefficient. We also prove that the effect of the error in the assumption on the initial blur is asymmetric and that the method is strongly degraded in the presence of aliasing or without a correct assumption on the camera blur. This analysis leads to a series of practical recommendations.     

Scale-Space Image Analysis Based on Hermite Polynomials Theory

The Hermite transform allows to locally approximate an image by a linear combination of polynomials. For a given scale σ and position ξ, the polynomial coefficients are closely related to the differential jet (set of partial derivatives of the blurred image) for the same scale and position. By making use of a classical formula due to Mehler (late 19th century), we establish a linear relationship linking the differential jets at two different scales σ and positions ξ involving Hermite polynomials. For multi-dimensional images, anisotropic excursions in scale-space can be handled in this way. Pattern registration and matching applications are suggested.We introduce a Gaussian windowed correlation function K (ν) for locally matching two images. When taking the mutual translation parameter ν as an independent variable, we express the Hermite coefficients of K (ν)interms of the Hermite coefficients of the two images being matched. This new result bears similarity with the Wiener-Khinchin theorem which links the Fourier transform of the conventional (flat-windowed) correlation function with the Fourier spectra of the images being correlated. Compared to the conventional correlation function, ours is more suited for matching localized image features.Numerical simulations using 2D test images illustrate the potentials of our proposals for signal and image matching in terms of accuracy and algorithmic complexity.First online version published in June, 2005     

基于尺度空间理论和反扩散函数的图像去噪

在使用扩散过程平滑噪声之后引入反扩散过程来恢复边缘,结合尺度空间理论和反扩散函数对图像进行去噪处理。该方法使用最小描述长度（MDL）准则自适应地选择图像中每一点处的最优尺度对图像进行滤波。加入尺度范围限制降低了过平滑和欠平滑的影响。改进了反扩散函数模型,对降质图像中的边缘进行恢复。与经典的滤波方法以及各向异性扩散方程的结果相比。本文方法取得了较好的效果。     

Perceptual Scale-Space and Its Applications

When an image is viewed at varying resolutions, it is known to create discrete perceptual jumps or transitions amid the continuous intensity changes. In this paper, we study a perceptual scale-space theory which differs from the traditional image scale-space theory in two aspects. (i) In representation, the perceptual scale-space adopts a full generative model. From a Gaussian pyramid it computes a sketch pyramid where each layer is a primal sketch representation (Guo et al. in Comput. Vis. Image Underst. 106(1):5–19, 2007)—an attribute graph whose elements are image primitives for the image structures. Each primal sketch graph generates the image in the Gaussian pyramid, and the changes between the primal sketch graphs in adjacent layers are represented by a set of basic and composite graph operators to account for the perceptual transitions. (ii) In computation, the sketch pyramid and graph operators are inferred, as hidden variables, from the images through Bayesian inference by stochastic algorithm, in contrast to the deterministic transforms or feature extraction, such as computing zero-crossings, extremal points, and inflection points in the image scale-space. Studying the perceptual transitions under the Bayesian framework makes it convenient to use the statistical modeling and learning tools for (a) modeling the Gestalt properties of the sketch graph, such as continuity and parallelism etc; (b) learning the most frequent graph operators, i.e. perceptual transitions, in image scaling; and (c) learning the prior probabilities of the graph operators conditioning on their local neighboring sketch graph structures. In experiments, we learn the parameters and decision thresholds through human experiments, and we show that the sketch pyramid is a more parsimonious representation than a multi-resolution Gaussian/Wavelet pyramid. We also demonstrate an application on adaptive image display—showing a large image in a small screen (say PDA) through a selective tour of its image pyramid. In this application, the sketch pyramid provides a means for calculating information gain in zooming-in different areas of an image by counting a number of operators expanding the primal sketches, such that the maximum information is displayed in a given number of frames. A short version was published in ICCV05 (Wang et al. 2005).     

Scale Selection Properties of Generalized Scale-Space Interest Point Detectors

Scale-invariant interest points have found several highly successful applications in computer vision, in particular for image-based matching and recognition. This paper presents a theoretical analysis of the scale selection properties of a generalized framework for detecting interest points from scale-space features presented in Lindeberg (Int. J. Comput. Vis. 2010, under revision) and comprising:

an enriched set of differential interest operators at a fixed scale including the Laplacian operator, the determinant of the Hessian, the new Hessian feature strength measures I and II and the rescaled level curve curvature operator, as well as

an enriched set of scale selection mechanisms including scale selection based on local extrema over scale, complementary post-smoothing after the computation of non-linear differential invariants and scale selection based on weighted averaging of scale values along feature trajectories over scale.

It is shown how the selected scales of different linear and non-linear interest point detectors can be analyzed for Gaussian blob models. Specifically it is shown that for a rotationally symmetric Gaussian blob model, the scale estimates obtained by weighted scale selection will be similar to the scale estimates obtained from local extrema over scale of scale normalized derivatives for each one of the pure second-order operators. In this respect, no scale compensation is needed between the two types of scale selection approaches. When using post-smoothing, the scale estimates may, however, be different between different types of interest point operators, and it is shown how relative calibration factors can be derived to enable comparable scale estimates for each purely second-order operator and for different amounts of self-similar post-smoothing. A theoretical analysis of the sensitivity to affine image deformations is presented, and it is shown that the scale estimates obtained from the determinant of the Hessian operator are affine covariant for an anisotropic Gaussian blob model. Among the other purely second-order operators, the Hessian feature strength measure I has the lowest sensitivity to non-uniform scaling transformations, followed by the Laplacian operator and the Hessian feature strength measure II. The predictions from this theoretical analysis agree with experimental results of the repeatability properties of the different interest point detectors under affine and perspective transformations of real image data. A number of less complete results are derived for the level curve curvature operator.     

Multi scale Harris corner detector based on Differential Morphological Decomposition

In this paper, a novel method for multi scale corner analysis and detection is presented. First, state-of-the-art Harris-Laplace corner detector is reminded, which benefits from linear scale-space analysis. Secondly, a non-linear scale-space transform, namely Differential Morphological Decomposition, is described. This multi-scale transform is used jointly with the Harris corner indicator to build a new multi scale corner detector. Both corner detectors are visually assessed on synthetic and satellite images, highlighting the advantages of such a method.     

Differential and Integral Geometry of Linear Scale-Spaces

Linear scale-space theory provides a useful framework to quantify the differential and integral geometry of spatio-temporal input images. In this paper that geometry comes about by constructing connections on the basis of the similarity jets of the linear scale-spaces and by deriving related systems of Cartan structure equations. A linear scale-space is generated by convolving an input image with Green's functions that are consistent with an appropriate Cauchy problem. The similarity jet consists of those geometric objects of the linear scale-space that are invariant under the similarity group. The constructed connection is assumed to be invariant under the group of Euclidean movements as well as under the similarity group. This connection subsequently determines a system of Cartan structure equations specifying a torsion two-form, a curvature two-form and Bianchi identities. The connection and the covariant derivatives of the curvature and torsion tensor then completely describe a particular local differential geometry of a similarity jet. The integral geometry obtained on the basis of the chosen connection is quantified by the affine translation vector and the affine rotation vectors, which are intimately related to the torsion two-form and the curvature two-form, respectively. Furthermore, conservation laws for these vectors form integral versions of the Bianchi identities. Close relations between these differential geometric identities and integral geometric conservation laws encountered in defect theory and gauge field theories are pointed out. Examples of differential and integral geometries of similarity jets of spatio-temporal input images are treated extensively.     

Scale-space derived from B-splines

This paper proposes a scale-space theory based on B-spline kernels. Our aim is twofold: 1) present a general framework, and show how B-splines provide a flexible tool to design various scale-space representations. In particular, we focus on the design of continuous scale-space and dyadic scale-space frame representations. A general algorithm is presented for fast implementation of continuous scale-space at rational scales. In the dyadic case, efficient frame algorithms are derived using B-spline techniques to analyze the geometry of an image. The relationship between several scale-space approaches is explored. The behavior of edge models, the properties of completeness, causality, and other properties in such a scale-space representation are examined in the framework of B-splines. It is shown that, besides the good properties inherited from the Gaussian kernel, the B-spline derived scale-space exhibits many advantages for modeling visual mechanism including the efficiency, compactness, orientation feature and parallel structure     

Morphological Counterparts of Linear Shift-Invariant Scale-Spaces

It is well-known that there are striking analogies between linear shift-invariant systems and morphological systems for image analysis. So far, however, the relations between both system theories are mainly understood on a pure convolution / erosion level. A formal connection on the level of differential or pseudodifferential equations and their induced scale-spaces is still missing. The goal of our paper is to close this gap. We present a simple and fairly general dictionary that allows to translate any linear shift-invariant evolution equation into its morphological counterpart and vice versa. It is based on a scale-space representation by means of the symbol of its (pseudo)differential operator. Introducing a novel transformation, the Cramér–Fourier transform, puts us in a position to relate the symbol to the structuring function of a morphological scale-space of Hamilton–Jacobi type. As an application of our general theory, we derive the morphological counterparts of many linear shift-invariant scale-spaces, such as the Poisson scale-space, \(\alpha \)-scale-spaces, summed \(\alpha \)-scale-spaces, relativistic scale-spaces, and their anisotropic variants. Our findings are illustrated by experiments.     

A Scale-Space Medialness Transform Based on Boundary Concordance Voting

The Concordance-based Medial Axis Transform (CMAT) presented in this paper is a multiscale medial axis (MMA) algorithm that computes the medial response from grey-level boundary measures. This non-linear operator responds only to symmetric structures, overcoming the limitations of linear medial operators which create side-lobe responses for symmetric structures and respond to edge structures. In addition, the spatial localisation of the medial axis and the identification of object width is improved in the CMAT algorithm compared with linear algorithms. The robustness of linear medial operators to noise is preserved in our algorithm. The effectiveness of the CMAT is accredited to the concordance property described in this paper. We demonstrate the performance of this method with test figures used by other authors and medical images that are relatively complex in structure. In these complex images the benefit of the improved response of our non-linear operator is clearly visible.     

Second Order Structure of Scale-Space Measurements

The second-order structure of random images f :? ^d →? ^N is studied under the assumption of stationarity of increments, isotropy and scale invariance. Scale invariance is defined via linear scale space theory. The results are formulated in terms of the covariance structure of the jet consisting of the scale space derivatives at a single point. Operators describing the effect in jet space of blurring and scaling are investigated. The theory developed is applicable in the analysis of naturally occurring images of which examples are provided.     

Exploring and exploiting the structure of saddle points in Gaussian scale space

When an image is filtered with a Gaussian of width σ and σ is considered as an extra dimension, the image is extended to a Gaussian scale-space (GSS) image. In earlier work it was shown that the GSS-image contains an intensity-based hierarchical structure that can be represented as a binary ordered rooted tree. Key elements in the construction of the tree are iso-intensity manifolds and scale-space saddles.A scale-space saddle is a critical point in scale space. When it connects two different parts of an iso-intensity manifold, it is called “dividing”, otherwise it is called “void”. Each dividing scale-space saddle is connected to an extremum in the original image via a curve in scale space containing critical points. Using the nesting of the iso-intensity manifolds in the GSS-image and the dividing scale-space saddles, each extremum is connected to another extremum. In the tree structure, the dividing scale-space saddles form the connecting elements in the hierarchy: they are the nodes of the tree. The extrema of the image form the leaves, while the critical curves are represented as the edges.To identify the dividing scale-space saddles, a global investigation of the scale-space saddles and the iso-intensity manifolds through them is needed.In this paper an overview of the situations that can occur is given. In each case it is shown how to distinguish between void and dividing scale-space saddles. Furthermore, examples are given, and the difference between selecting the dividing and the void scale-space saddles is shown. Also relevant geometric properties of GSS images are discussed, as well as their implications for algorithms used for the tree extraction.As main result, it is not necessary to search through the whole GSS image to find regions related to each relevant scale-space saddle. This yields a considerable reduction in complexity and computation time, as shown in two examples.