Global plus local: A complete framework for feature extraction and recognition

Linear discriminant analysis (LDA) is one of the most popular supervised feature extraction techniques used in machine learning and pattern classification. However, LDA only captures global geometrical structure information of the data and ignores the geometrical structure information of local data points. Though many articles have been published to address this issue, most of them are incomplete in the sense that only part of the local information is used. We show here that there are total three kinds of local information, namely, local similarity information, local intra-class pattern variation, and local inter-class pattern variation. We first propose a new method called enhanced within-class LDA (EWLDA) algorithm to incorporate the local similarity information, and then propose a complete framework called complete global–local LDA (CGLDA) algorithm to incorporate all these three kinds of local information. Experimental results on two image databases demonstrate the effectiveness of our algorithms.     

A regularization framework for robust dimensionality reduction with applications to image reconstruction and feature extraction

Dimensionality reduction has many applications in pattern recognition, machine learning and computer vision. In this paper, we develop a general regularization framework for dimensionality reduction by allowing the use of different functions in the cost function. This is especially important as we can achieve robustness in the presence of outliers. It is shown that optimizing the regularized cost function is equivalent to solving a nonlinear eigenvalue problem under certain conditions, which can be handled by the self-consistent field (SCF) iteration. Moreover, this regularization framework is applicable in unsupervised or supervised learning by defining the regularization term which provides some types of prior knowledge of projected samples or projected vectors. It is also noted that some linear projection methods can be obtained from this framework by choosing different functions and imposing different constraints. Finally, we show some applications of our framework by various data sets including handwritten characters, face images, UCI data, and gene expression data.     

Classification in a normalized feature space using support vector machines

This paper discusses classification using support vector machines in a normalized feature space. We consider both normalization in input space and in feature space. Exploiting the fact that in this setting all points lie on the surface of a unit hypersphere we replace the optimal separating hyperplane by one that is symmetric in its angles, leading to an improved estimator. Evaluation of these considerations is done in numerical experiments on two real-world datasets. The stability to noise of this offset correction is subsequently investigated as well as its optimality.     

Nonlinear feature extraction and classification of multivariate data in kernel feature space

Batch processes have played an essential role in the production of high value-added product of chemical, pharmaceutical, food, bio-chemical, and semi-conductor industries. For productivity and quality improvement, several multivariate statistical techniques such as principal component analysis (PCA) and Fisher discriminant analysis (FDA) have been developed to solve a fault diagnosis problem of batch processes. Fisher discriminant analysis, as a traditional statistical technique for feature extraction and classification, has been shown to be a good linear technique for fault diagnosis and outperform PCA based diagnosis methods. This paper proposes a more efficient nonlinear diagnosis method for batch processes using a kernel version of Fisher discriminant analysis (KFDA). A case study on two batch processes has been conducted. In addition, the diagnosis performance of the proposed method was compared with that of an existing diagnosis method based on linear FDA. The diagnosis results showed that the proposed KFDA based diagnosis method outperforms the linear FDA based method.     

Flow feature analysis and extraction for classifying axisymmetric vector field patterns

A relatively fixed structure vector field which includes clockwise vortex, anticlockwise vortex, convergence, divergence, and saddle is defined as axisymmetric vector fields (AVF) in this study. A method for characterizing and classifying the type of flow patterns in 2-dimension actual vector field for the meteorology application is proposed. First of all, the collected AVF samples are transformed to a directional pseudo-color image by means of mapping them to the hue component space in the HSL color model. Secondly, the directional hue difference and similarity degree to the normal AVF are respectively extracted as two features by the technique of image processing. Thirdly, two improved physical properties (vorticity and divergence) of AVF are introduced and improved for this study. Finally, in the experiment, the probability density distribution for every type of AVF samples is employed to analyze the four features advantages and disadvantages on the five AVF patterns classification. The correlation of the features is discussed by the PCA method. The statistics results show that the features are effective to describe the AVF patterns. By training a classifier based on constructing a decision tree, the classification ability of the features is proved on processing different scale and resolution AVF samples by comparing with traditional methods.     

A framework for design of scheduled output feedback model predictive control

We present a stabilizing scheduled output feedback Model Predictive Control (MPC) algorithm for constrained nonlinear systems with large operating regions. We design a set of local output feedback predictive controllers with their estimated regions of stability covering the desired operating region, and implement them as a single scheduled output feedback MPC which on-line switches between the set of local controllers and achieves nonlinear transitions with guaranteed stability. This algorithm provides a general framework for scheduled output feedback MPC design.     

Complete discriminant evaluation and feature extraction in kernel space for face recognition

This work proposes a method to decompose the kernel within-class eigenspace into two subspaces: a reliable subspace spanned mainly by the facial variation and an unreliable subspace due to limited number of training samples. A weighting function is proposed to circumvent undue scaling of eigenvectors corresponding to the unreliable small and zero eigenvalues. Eigenfeatures are then extracted by the discriminant evaluation in the whole kernel space. These efforts facilitate a discriminative and stable low-dimensional feature representation of the face image. Experimental results on FERET, ORL and GT databases show that our approach consistently outperforms other kernel based face recognition methods.

A vector space model approach to production planning for a multi-product,multi-cell and multi-stage manufacturing system

This paper develops a formal mathematical approach to aggregate production planning for a multi-product, multi-cell and multi-stage manufacturing system. The model, based upon a vector space approach, includes all the important variables relating to the demand for individual items, inventory levels, the availability of machines taking into account any breakdowns, subcontracting of orders and overtime working. The computational procedure for determining the production planning strategies, in terms of overtime/undertime working and increase/decrease in the number of orders subcontracted, are presented. Three numerical examples are presented showing the use of the model developed. This approach makes it possible to develop realistic models of practical manufacturing systems. It is particularly applicable to flexible manufacturing systems.     

Kernel-view based discriminant approach for embedded feature extraction in high-dimensional space

Derived from the traditional manifold learning algorithms, local discriminant analysis methods identify the underlying submanifold structures while employing discriminative information for dimensionality reduction. Mathematically, they can all be unified into a graph embedding framework with different construction criteria. However, such learning algorithms are limited by the curse-of-dimensionality if the original data lie on the high-dimensional manifold. Different from the existing algorithms, we consider the discriminant embedding as a kernel analysis approach in the sample space, and a kernel-view based discriminant method is proposed for the embedded feature extraction, where both PCA pre-processing and the pruning of data can be avoided. Extensive experiments on the high-dimensional data sets show the robustness and outstanding performance of our proposed method.     

KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition

This paper examines the theory of kernel Fisher discriminant analysis (KFD) in a Hilbert space and develops a two-phase KFD framework, i.e., kernel principal component analysis (KPCA) plus Fisher linear discriminant analysis (LDA). This framework provides novel insights into the nature of KFD. Based on this framework, the authors propose a complete kernel Fisher discriminant analysis (CKFD) algorithm. CKFD can be used to carry out discriminant analysis in "double discriminant subspaces." The fact that, it can make full use of two kinds of discriminant information, regular and irregular, makes CKFD a more powerful discriminator. The proposed algorithm was tested and evaluated using the FERET face database and the CENPARMI handwritten numeral database. The experimental results show that CKFD outperforms other KFD algorithms.     

Bayesian shape model for facial feature extraction and recognition

A facial feature extraction algorithm using the Bayesian shape model (BSM) is proposed in this paper. A full-face model consisting of the contour points and the control points is designed to describe the face patch, using which the warping/normalization of the extracted face patch can be performed efficiently. First, the BSM is utilized to match and extract the contour points of a face. In BSM, the prototype of the face contour can be adjusted adaptively according to its prior distribution. Moreover, an affine invariant internal energy term is introduced to describe the local shape deformations between the prototype contour in the shape domain and the deformable contour in the image domain. Thus, both global and local shape deformations can be tolerated. Then, the control points are estimated from the matching result of the contour points based on the statistics of the full-face model. Finally, the face patch is extracted and normalized using the piece-wise affine triangle warping algorithm. Experimental results based on real facial feature extraction demonstrate that the proposed BSM facial feature extraction algorithm is more accurate and effective as compared to that of the active shape model (ASM).     

A multi-objective genetic algorithm for simultaneous model and feature selection for support vector machines

The Support Vector Machines (SVM) constitute a very powerful technique for pattern classification problems. However, its efficiency in practice depends highly on the selection of the kernel function type and relevant parameter values. Selecting relevant features is another factor that can also impact the performance of SVM. The identification of the best set of parameters values for a classification model such as SVM is considered as an optimization problem. Thus, in this paper, we aim to simultaneously optimize SVMs parameters and feature subset using different kernel functions. We cast this problem as a multi-objective optimization problem, where the classification accuracy, the number of support vectors, the margin and the number of selected features define our objective functions. To solve this optimization problem, a method based on multi-objective genetic algorithm NSGA-II is suggested. A multi-criteria selection operator for our NSGA-II is also introduced. The proposed method is tested on some benchmark data-sets. The experimental results show the efficiency of the proposed method where features were reduced and the classification accuracy has been improved.     

A unified framework for document restoration using inpainting and shape-from-shading

We present a restoration framework to reduce undesirable distortions in imaged documents. Our framework is based on two components: (1) an image inpainting procedure that can separate non-uniform illumination (and other) artifacts from the printed content and (2) a shape-from-shading (SfS) formulation that can reconstruct the 3D shape of the document's surface. Used either piecewise or in its entirety, this framework can correct a variety of distortions including shading, shadow, ink-bleed, show-through, perspective and geometric distortions, for both camera-imaged and flatbed-imaged documents. Our overall framework is described in detail. In addition, our SfS formulation can be easily modified to target various illumination conditions to suit different real-world applications. Results on images of synthetic and real documents demonstrate the effectiveness of our approach. OCR results are also used to gauge the performance of our approach.     

A vector radiative transfer model for sea-surface salinity retrieval from space: a non-raining case

ABSTRACT

Sea-surface salinity (SSS) can be measured from space using a microwave sensor. However, achieving the desired accuracy in SSS retrieval is challenging due to the lower sensitivity of the brightness temperature to SSS especially at low sea-surface temperature conditions. The retrieval accuracy can be further degraded due to the atmospheric and sea-surface effects (including emission and reflection), which require more accurate correction methods based on the radiative transfer model. In this article, a vector radiative transfer model (VRTM) was developed based on a matrix operator method that considers the ocean–atmosphere system under non-raining conditions. The results from this model were compared with measurement data provided by the Soil Moisture and Ocean Salinity (SMOS) satellite sensor and the results from two other RT models (RT4 model and a forward model of the European Space Agency, ESA). Statistical evaluation of these results revealed that estimation errors of top of atmosphere (TOA) radiance by the VRTM model was less than 0.3% as compared to the RT4 model results. The difference of the brightness temperatures predicted by the VRTM model and measured by the SMOS was within 1.5 K which was better than the ESA’s forward model predictions. These results suggest that the VRTM is relatively more accurate and has high computational efficiency for simulating the TOA brightness temperature for various scientific research and remote-sensing applications.     

Evaluation of keyphrase extraction algorithm and tiling process for a document/resource recommender within e-learning environments

The research presented in this paper is an examination of the applicability of IUI techniques in an online e-learning environment. In particular we make use of user modeling techniques, information retrieval and extraction mechanisms and collaborative filtering methods. The domains of e-learning, web-based training and instruction and intelligent tutoring systems provide a challenging environment due to the large and diverse user population it entails. The overall system concentrates on utilizing a user modeling system to filter results as part of a collaborative document recommendation system. The goal of such a system is to actively seek out and recommend documents that will either encourage the users to expand their knowledge of a given topic or reinforce the knowledge which they already have. The system aims to recommend these documents in a non-intrusive manner with minimal user inconvenience, and attempts to do so by utilizing the Key Extraction Algorithm and automatically extracting queries, searching the web and filtering the search results. Users are encouraged to provide feedback about the resources and links they have viewed.     

Conceptual framework for document semantic modelling: an application to document and knowledge management in the legal domain

The main contribution of this paper is to lay down a conceptual framework for document semantics modeling. This framework provides a generic graphical knowledge representation model based on Sowa’s conceptual structures. Modeling primitives are introduced to represent factual and ontological knowledge that can be expressed in electronic documents. Binding features are proposed so as to keep knowledge representation and knowledge formulation linked together.

This framework may be applied to various domains and may accept, for this purpose, many different ontological extensions. Thus an extension is provided so as to properly handle the particular kind of knowledge encountered in the legal domain.