Optimal approximation of signal priors

In signal restoration by Bayesian inference, one typically uses a parametric model of the prior distribution of the signal. Here, we consider how the parameters of a prior model should be estimated from observations of uncorrupted signals. A lot of recent work has implicitly assumed that maximum likelihood estimation is the optimal estimation method. Our results imply that this is not the case. We first obtain an objective function that approximates the error occurred in signal restoration due to an imperfect prior model. Next, we show that in an important special case (small gaussian noise), the error is the same as the score-matching objective function, which was previously proposed as an alternative for likelihood based on purely computational considerations. Our analysis thus shows that score matching combines computational simplicity with statistical optimality in signal restoration, providing a viable alternative to maximum likelihood methods. We also show how the method leads to a new intuitive and geometric interpretation of structure inherent in probability distributions.     

Equivalence of some common linear feature extraction techniques for appearance-based object recognition tasks

Recently, a number of empirical studies have compared the performance of PCA and ICA as feature extraction methods in appearance-based object recognition systems, with mixed and seemingly contradictory results. In this paper, we briefly describe the connection between the two methods and argue that whitened PCA may yield identical results to ICA in some cases. Furthermore, we describe the specific situations in which ICA might significantly improve on PCA     

Consistency of pseudolikelihood estimation of fully visible Boltzmann machines

A Boltzmann machine is a classic model of neural computation, and a number of methods have been proposed for its estimation. Most methods are plagued by either very slow convergence or asymptotic bias in the resulting estimates. Here we consider estimation in the basic case of fully visible Boltzmann machines. We show that the old principle of pseudolikelihood estimation provides an estimator that is computationally very simple yet statistically consistent.     

Topography as a property of the natural sensory world

The topographic organization of brain areas such as theprimary visual cortex is usually assumed to reflect exclusivelyanatomic constraints like wiring length. Here we argue thattopography is in fact imbedded in the statistical structure of thenatural sensory input. Thus, the topography of sensory areasreflects the statistics of its input as well, in the same way as thesparseness of cell outputs.     

Estimating Overcomplete Independent Component Bases for Image Windows

Estimating overcomplete ICA bases for image windows is a difficult problem. Most algorithms require the estimation of values of the independent components which leads to computationally heavy procedures. Here we first review the existing methods, and then introduce two new algorithms that estimate an approximate overcomplete basis quite fast in a high-dimensional space. The first algorithm is based on the prior assumption that the basis vectors are randomly distributed in the space, and therefore close to orthogonal. The second replaces the conventional orthogonalization procedure by a transformation of the marginal density to gaussian.     

The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis

The author previously introduced a fast fixed-point algorithm for independent component analysis. The algorithm was derived from objective functions motivated by projection pursuit. In this paper, it is shown that the algorithm is closely connected to maximum likelihood estimation as well. The basic fixed-point algorithm maximizes the likelihood under the constraint of decorrelation, if the score function is used as the nonlinearity. Modifications of the algorithm maximize the likelihood without constraints.     

Simple-cell-like receptive fields maximize temporal coherence in natural video

Recently, statistical models of natural images have shown the emergence of several properties of the visual cortex. Most models have considered the nongaussian properties of static image patches, leading to sparse coding or independent component analysis. Here we consider the basic time dependencies of image sequences instead of their nongaussianity. We show that simple-cell-type receptive fields emerge when temporal response strength correlation is maximized for natural image sequences. Thus, temporal response strength correlation, which is a nonlinear measure of temporal coherence, provides an alternative to sparseness in modeling simple-cell receptive field properties. Our results also suggest an interpretation of simple cells in terms of invariant coding principles, which have previously been used to explain complex-cell receptive fields.     

Complex cell pooling and the statistics of natural images

In previous work, we presented a statistical model of natural images that produced outputs similar to receptive fields of complex cells in primary visual cortex. However, a weakness of that model was that the structure of the pooling was assumed a priori and not learned from the statistical properties of natural images. Here, we present an extended model in which the pooling nonlinearity and the size of the subspaces are optimized rather than fixed, so we make much fewer assumptions about the pooling. Results on natural images indicate that the best probabilistic representation is formed when the size of the subspaces is relatively large, and that the likelihood is considerably higher than for a simple linear model with no pooling. Further, we show that the optimal nonlinearity for the pooling is squaring. We also highlight the importance of contrast gain control for the performance of the model. Our model is novel in that it is the first to analyze optimal subspace size and how this size is influenced by contrast normalization.     

The sustainability challenge of meeting carbon dioxide targets in Europe by 2020

Following the Kyoto Protocol, the European Union obligated itself to lower its greenhouse gas (GHG) emissions 20% below their 1990 level, by the year 2020. Carbon dioxide is the major GHG. To fulfil this obligation, the nations must meet the sustainability challenge of countering rising population plus affluence with the dematerialization of less energy per GDP plus the decarbonization of less carbon per energy. To test the feasibility of meeting the challenge, we analysed carbon dioxide emission during 1993–2004. Although emissions in the entire Union grew only by an average of 0.31% per year, emissions and their drivers varied markedly among the 27 member states. Dematerialization and decarbonization did occur, but not enough to offset the slight population growth plus rapidly increasing affluence. To fulfil its obligation in the next 12 years, the EU27 would have to counter its increasing population and affluence by a combined dematerialization and decarbonization 1.9–2.6 times faster than during 1993–2004. Hence, fulfilling its obligation by addressing fossil carbon emissions alone is very unlikely.