Adaptive face representation via class-specific and intra-class variation dictionaries for recognition

Face recognition has attracted extensive interests due to its wide applications. However, there are many challenges in the real world scenario. For example, relatively few samples are available for training. Face images collected from surveillance cameras may consist of complex variations (e.g. illumination, expression, occlusion and pose). To address these challenges, in this paper we propose learning class-specific and intra-class variation dictionaries separately. Specifically, we first develop a discriminative class-specific dictionary amplifying the differences between training classes. We impose a constraint on sparse coefficients, which guarantees the sparse representation coefficients having small within-class scatter and large between-class scatter. Moreover, we introduce a new intra-class variation dictionary based on the assumption that similar variations from different classes may share some common features. The intra-class variation dictionary not only captures the inner-relationship of variations, but also addresses the limitation of the manually designed dictionaries that are person-specific. Finally, we apply the combined dictionary to adaptively represent face images. Experiments conducted on the AR, CMU-PIE, FERET and Extended Yale B databases show the effectiveness of the proposed method.     

3D face recognition with sparse spherical representations

This paper addresses the problem of 3D face recognition using simultaneous sparse approximations on the sphere. The 3D face point clouds are first aligned with a fully automated registration process. They are then represented as signals on the 2-sphere in order to preserve depth and geometry information. Next, we implement a dimensionality reduction process with simultaneous sparse approximations and subspace projection. It permits to represent each 3D face by only a few spherical functions that are able to capture the salient facial characteristics, and hence to preserve the discriminant facial information. We eventually perform recognition by effective matching in the reduced space, where linear discriminant analysis can be further activated for improved recognition performance. The 3D face recognition algorithm is evaluated on the FRGC v.1.0 data set, where it is shown to outperform classical state-of-the-art solutions that work with depth images.     

3D surface filtering using spherical harmonics

This paper presents a novel approach for 3D surface filtering over two-manifold meshes. A robust spherical parameterization algorithm is proposed to transform the input surface into a spherical vector function/signal. This signal is then decomposed into frequency domain using spherical harmonic transforms. Finally, traditional filtering techniques are generalized to process such spherical signals in either the frequency or spatial domain. Our major contribution is the two-phase spherical parameterization algorithm, which can handle meshes with complex shapes by incorporating local parameterization into the progressive mesh. A number of experimental examples demonstrate the potential of our algorithm.     

3D free-form surface registration and object recognition

A new technique to recognise 3D free-form objects via registration is proposed. This technique attempts to register a free-form surface, represented by a set of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr% 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9% vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x% fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaGaaGOmamaala% aabaGaaGymaaqaaiaaikdaaaGaamiraaaa!38F8!\[2\frac{1}{2}D\] sensed data points, to the model surface, represented by another set of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr% 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9% vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x% fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaGaaGOmamaala% aabaGaaGymaaqaaiaaikdaaaGaamiraaaa!38F8!\[2\frac{1}{2}D\] model data points, without prior knowledge of correspondence or view points between the two point sets. With an initial assumption that the sensed surface be part of a more complete model surface, the algorithm begins by selecting three dispersed, reliable points on the sensed surface. To find the three corresponding model points, the method uses the principal curvatures and the Darboux frames to restrict the search over the model space. Invariably, many possible model 3-typles will be found. For each hypothesized model 3-tuple, the transformation to match the sensed 3-tuple to the model 3-tuple can be determined. A heuristic search is proposed to single out the optimal transformation in low order time. For realistic object recognition or registration, where the two range images are often extracted from different view points of the model, the earlier assumption that the sensed surface be part of a more complete model surface cannot be relied on. With this, the sensed 3-tuple must be chosen such that the three sensed points lie on the common region visible to both the sensed and model views. We propose an algorithm to select a minimal non-redundant set of 3-tuples such that at least one of the 3-tuples will lie on the overlap. Applying the previous algorithm to each 3-tuple within this set, the optimal transformation can be determined. Experiments using data obtained from a range finder have indicated fast registration for relatively complex test cases. If the optimal registrations between the sensed data (candidate) and each of a set of model data are found, then, for 3D object recognition purposes, the minimal best fit error can be used as the decision rule.     

Face relighting using discriminative 2D spherical spaces for face recognition

As part of the face recognition task in a robust security system, we propose a novel approach for the illumination recovery of faces with cast shadows and specularities. Given a single 2D face image, we relight the face object by extracting the nine spherical harmonic bases and the face spherical illumination coefficients by using the face spherical spaces properties. First, an illumination training database is generated by computing the properties of the spherical spaces out of face albedo and normal values estimated from 2D training images. The training database is then discriminately divided into two directions in terms of the illumination quality and light direction of each image. Based on the generated multi-level illumination discriminative training space, we analyze the target face pixels and compare them with the appropriate training subspace using pre-generated tiles. When designing the framework, practical real-time processing speed and small image size were considered. In contrast to other approaches, our technique requires neither 3D face models nor restricted illumination conditions for the training process. Furthermore, the proposed approach uses one single face image to estimate the face albedo and face spherical spaces. In this work, we also provide the results of a series of experiments performed on publicly available databases to show the significant improvements in the face recognition rates.     

CUDA-enabled implementation of a neural network algorithm for handwritten digit recognition

Using a convolutional neural network as an example, we discuss specific aspects of implementing a learning algorithm of pattern recognition on the GPU graphics card using NVIDIA CUDA architecture. The training time of the neural network on a video-adapter is decreased by a factor of 5.96 and the recognition time of a test set is decreased by a factor of 8.76 when compared with the implementation of an optimized algorithm on a central processing unit (CPU). We show that the implementation of the neural network algorithms on graphics processors holds promise.     

Make3D: Learning 3D Scene Structure from a Single Still Image

We consider the problem of estimating detailed 3D structure from a single still image of an unstructured environment. Our goal is to create 3D models that are both quantitatively accurate as well as visually pleasing. For each small homogeneous patch in the image, we use a Markov Random Field (MRF) to infer a set of plane parameters” that capture both the 3D location and 3D orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3D structure than does prior art and also give a much richer experience in the 3D flythroughs created using image-based rendering, even for scenes with significant nonvertical structure. Using this approach, we have created qualitatively correct 3D models for 64.9 percent of 588 images downloaded from the Internet. We have also extended our model to produce large-scale 3D models from a few images.     

3D floor plan recovery from overlapping spherical images

Computational Visual Media - We present a novel approach to automatically recover, from a small set of partially overlapping spherical images, an indoor structure representation in terms of a 3D...     

FPGA acceleration on a multi-layer perceptron neural network for digit recognition

The Journal of Supercomputing - This paper proposes field-programmable gate array (FPGA) acceleration on a scalable multi-layer perceptron (MLP) neural network for classifying handwritten digits....     

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a realistic modification of its 3D model. In this paper, we show that such information can be learned from user‐clicked 2D images and a template 3D model of the target animal. We present a volumetric deformation framework that produces a set of new 3D models by deforming a template 3D model according to a set of user‐clicked images. Our framework is based on a novel locally‐bounded deformation energy, where every local region has its own stiffness value that bounds how much distortion is allowed at that location. We jointly learn the local stiffness bounds as we deform the template 3D mesh to match each user‐clicked image. We show that this seemingly complex task can be solved as a sequence of convex optimization problems. We demonstrate the effectiveness of our approach on cats and horses, which are highly deformable and articulated animals. Our framework produces new 3D models of animals that are significantly more plausible than methods without learned stiffness.     

Model-based recognition using 3D structure from motion

The effectiveness of shape information alone, without size, for recognizing stored 3D models is considered. The geometric constraint filtering method of Grimson and Lozano-Pérez is used to curb the potentially combinatorial expansion of the model search space. Results, typical of those from several models experimented with, are given for the task of recognizing a plug from uncertain surface normal data. They show that, at least for an ‘interesting’ view, shape data is highly effective, even when the sensed surface normals are uncertain in direction to, say, ± 10°. The loss of size information does, however, result in a drop in search efficiency, but this appears relatively small: u factor of ≈ 5 for the example of a plug at the 10° error mark.     

Automated surface micro-machining mask creation from a 3D model

We have developed and implemented a method, which given a three-dimensional object can infer from topology the two-dimensional masks needed to produce that object with surface micromachining. The masks produced by this design tool can be generic, process independent masks, or if given process constraints, specific for a target process. This design tool calculates the two-dimensional mask set required to produce a given three-dimensional model by investigating the vertical topology to the model. The 3D model is first separated into bodies that are non-intersecting, made from different materials or only linked through a ground plane. Next, for each body unique vertical cross sections are located and arranged into a tree based on their topological relationship. A branch-wise search of the tree uncovers locations where deposition boundaries must lie and identifies candidate masks creating a generic mask set for the 3D model. Finally, in the last step specific process requirements are considered that may constrain the generic mask set. Constraints can include the thickness or number of deposition layers, specific ordering of masks as required by a process and type of material used in a given layer. Candidate masks are reconciled with the process constraints through a constrained optimization.     

Model-based recognition of 3D objects from single images

In this work, we treat major problems of object recognition which have received relatively little attention lately. Among them are the loss of depth information in the projection from a 3D object to a single 2D image, and the complexity of finding feature correspondences between images. We use geometric invariants to reduce the complexity of these problems. There are no geometric invariants of a projection from 3D to 2D. However, given certain modeling assumptions about the 3D object, such invariants can be found. The modeling assumptions can be either a particular model or a generic assumption about a class of models. Here, we use such assumptions for single-view recognition. We find algebraic relations between the invariants of a 3D model and those of its 2D image under general projective projection. These relations can be described geometrically as invariant models in a 3D invariant space, illuminated by invariant “light rays,” and projected onto an invariant version of the given image. We apply the method to real images     

Efficient 3D reconstruction for face recognition

Face recognition with variant pose, illumination and expression (PIE) is a challenging problem. In this paper, we propose an analysis-by-synthesis framework for face recognition with variant PIE. First, an efficient two-dimensional (2D)-to-three-dimensional (3D) integrated face reconstruction approach is introduced to reconstruct a personalized 3D face model from a single frontal face image with neutral expression and normal illumination. Then, realistic virtual faces with different PIE are synthesized based on the personalized 3D face to characterize the face subspace. Finally, face recognition is conducted based on these representative virtual faces. Compared with other related work, this framework has following advantages: (1) only one single frontal face is required for face recognition, which avoids the burdensome enrollment work; (2) the synthesized face samples provide the capability to conduct recognition under difficult conditions like complex PIE; and (3) compared with other 3D reconstruction approaches, our proposed 2D-to-3D integrated face reconstruction approach is fully automatic and more efficient. The extensive experimental results show that the synthesized virtual faces significantly improve the accuracy of face recognition with changing PIE.