Factorization with Uncertainty

Factorization using Singular Value Decomposition (SVD) is often used for recovering 3D shape and motion from feature correspondences across multiple views. SVD is powerful at finding the global solution to the associated least-square-error minimization problem. However, this is the correct error to minimize only when the x and y positional errors in the features are uncorrelated and identically distributed. But this is rarely the case in real data. Uncertainty in feature position depends on the underlying spatial intensity structure in the image, which has strong directionality to it. Hence, the proper measure to minimize is covariance-weighted squared-error (or the Mahalanobis distance). In this paper, we describe a new approach to covariance-weighted factorization, which can factor noisy feature correspondences with high degree of directional uncertainty into structure and motion. Our approach is based on transforming the raw-data into a covariance-weighted data space, where the components of noise in the different directions are uncorrelated and identically distributed. Applying SVD to the transformed data now minimizes a meaningful objective function in this new data space. This is followed by a linear but suboptimal second step to recover the shape and motion in the original data space. We empirically show that our algorithm gives very good results for varying degrees of directional uncertainty. In particular, we show that unlike other SVD-based factorization algorithms, our method does not degrade with increase in directionality of uncertainty, even in the extreme when only normal-flow data is available. It thus provides a unified approach for treating corner-like points together with points along linear structures in the image.     

Static and dynamic information derived from source and system features for person recognition from humming

In this paper, hum of a person (instead of normal speech) is used to design a voice biometric system for person recognition. In addition, a recently proposed static feature set, viz., Variable length Teager energy based Mel Frequency Cepstral Coefficients (VTMFCC), is found to capture source-like information of a hum signal. Effectiveness of VTMFCC over linear prediction (LP) residual to capture the complementary information than MFCC is demonstrated in a hum signal. Person recognition performance is found to be better when a score-level fusion is used by combining evidences from static and dynamic features for MFCC (system) and VTMFCC (source-like) features than MFCC alone. Experiments are validated on two types of dynamic features, viz., delta cepstrum and shifted delta cepstrum. In addition, for score-level fusion using static and dynamic features % identification rate and % Equal Error Rate are observed to outperform by 7.9?% and 0.27?%, respectively than MFCC alone. Furthermore, we have observed that person recognition system gives better performance for larger frame duration 69.6?ms as opposed to traditional 10–30?ms frame duration.     

A Fuzzy Statistical Rule Generation Method for Handwriting Recognition

This paper presents a statistical approach for rule-base generation of handwriting recognition. The proposed method integrates the heuristic feature selection with the statistical evaluation and thus improves the performance of the rule generation as well as of the fuzzy handwriting recognition system. Fuzzy statistical measures are employed to identify relevant features from a given large handwriting database. First an automatic rule-base mechanism is presented. To reduce the time needed for this generation mechanism an additional heuristic feature selection step is introduced. Tests show that this generated rule-base improved the recognition results over previous approaches.     

A hardware solution for real-time intelligent fingerprint acquisition

The first step in any fingerprint recognition system is the fingerprint acquisition. A well-acquired fingerprint image results in high-resolution accuracy and low computational effort of processing. Hence, it is very useful for the recognition system to evaluate recognition confidence level to request new fingerprint samples if the confidence level is low, and to facilitate recognition process if the confidence level is high. This paper presents a hardware solution to ensure a successful and friendly acquisition of the fingerprint image, which can be incorporated at low cost into an embedded fingerprint recognition system due to its small size and high speed. The solution implements a novel technique based on directional image processing that allows not only the estimation of fingerprint image quality, but also the extraction of useful information (in particular, singular points). The digital architecture of the module is detailed and their features in terms of resource consumption and processing speed are illustrated with implementation results into FPGAs from Xilinx. Performance of the solution has been verified with fingerprints from several standard databases that have been acquired with sensors of different sizes and technologies (optical, capacitive, and thermal sweeping).     

On the estimation of a large sparse Bayesian system: The Snaer program

The Snaer program calculates the posterior mean and variance of variables on some of which we have data (with precisions), on some we have prior information (with precisions), and on some prior indicator ratios (with precisions) are available. The variables must satisfy a number of exact restrictions. The system is both large and sparse. Two aspects of the statistical and computational development are a practical procedure for solving a linear integer system, and a stable linearization routine for ratios. The numerical method for solving large sparse linear least-squares estimation problems is tested and found to perform well, even when the n×k design matrix is large (nk=O(10⁸)).     

Random forests and the data sparseness problem in language modeling

Language modeling is the problem of predicting words based on histories containing words already hypothesized. Two key aspects of language modeling are effective history equivalence classification and robust probability estimation. The solution of these aspects is hindered by the data sparseness problem.Application of random forests (RFs) to language modeling deals with the two aspects simultaneously. We develop a new smoothing technique based on randomly grown decision trees (DTs) and apply the resulting RF language models to automatic speech recognition. This new method is complementary to many existing ones dealing with the data sparseness problem. We study our RF approach in the context of n-gram type language modeling in which n − 1 words are present in a history. Unlike regular n-gram language models, RF language models have the potential to generalize well to unseen data, even when histories are longer than four words. We show that our RF language models are superior to the best known smoothing technique, the interpolated Kneser–Ney smoothing, in reducing both the perplexity (PPL) and word error rate (WER) in large vocabulary state-of-the-art speech recognition systems. In particular, we will show statistically significant improvements in a contemporary conversational telephony speech recognition system by applying the RF approach only to one of its many language models.     

ILP heuristics and a new exact method for bi-objective 0/1 ILPs: Application to FTTx-network design

Heuristics and metaheuristics are inevitable ingredients of most of the general purpose ILP solvers today, because of their contribution to the significant boost of the performance of exact methods. In the field of bi/multi-objective optimization, to the best of our knowledge, it is still not very common to integrate ILP heuristics into exact solution frameworks. This paper aims to bring a stronger attention of both the exact and metaheuristic communities to still unexplored possibilities for performance improvements of exact and heuristic multi-objective optimization algorithms.We focus on bi-objective optimization problems whose feasible solutions can be described as 0/1 integer linear programs and propose two ILP heuristics, boundary induced neighborhood search (BINS) and directional local branching. Their main idea is to combine the features and explore the neighborhoods of solutions that are relatively close in the objective space. A two-phase ILP-based heuristic framework relying on BINS and directional local branching is introduced. Moreover, a new exact method called adaptive search in objective space (ASOS) is also proposed. ASOS combines features of the ϵ-constraint method with the binary search in the objective space and uses heuristic solutions produced by BINS for guidance. Our new methods are computationally evaluated on two problems of particular relevance for the design of FTTx-networks. Comparison with other known exact methods (relying on the exploration of the objective space) is conducted on a set of realistic benchmark instances representing telecommunication access networks from Germany.     

Ontology-based context modeling for emotion recognition in an intelligent web

We describe an ontological model for representation and integration of electroencephalographic (EEG) data and apply it to detect human emotional states. The model (BIO_EMOTION) is an ontology-based context model for emotion recognition and acts as a basis for: (1) the modeling of users’ contexts, including user profiles, EEG data, the situation and environment factors, and (2) supporting reasoning on the users’ emotional states. Because certain ontological concepts in the EEG domain are ill-defined, we formally represent and store these concepts, their taxonomies and high-level representation (i.e., rules) in the model. To evaluate the effectiveness for inferring emotional states, DEAP dataset is used for model reasoning. Result shows that our model reaches an average recognition ratio of 75.19 % on Valence and 81.74 % on Arousal for eight participants. As mentioned above, the BIO-EMOTION model acts like a bridge between users’ emotional states and low-level bio-signal features. It can be integrated in user modeling techniques, and be used to model web users’ emotional states in human-centric web aiming to provide active, transparent, safe and reliable services to users. This work aims at, in other words, creating an ontology-based context model for emotion recognition using EEG. Particularly, this model completely implements the loop body of the W2T data cycle once: from low-level EEG feature acquisition to emotion recognition. A long-term goal for the study is to complete this model to implement the whole W2T data cycle.     

基于HMM和GMM的维吾尔语联机手写体识别研究

给出了一个基于HMM和GMM双引擎识别模型的维吾尔语联机手写体整词识别系统。在GMM部分,系统提取了8-方向特征,生成8-方向特征样式图像、定位空间采样点以及提取模糊的方向特征。在对模型精细化迭代训练之后,得到GMM模型文件。HMM部分,系统采用了笔段特征的方法来获取笔段分段点特征序列,在对模型进行精细化迭代训练后,得到HMM模型文件。将GMM模型文件和HMM模型文件分别打包封装再进行联合封装成字典。在第一期的实验中,系统的识别率达到97%,第二期的实验中,系统的识别率高达99%。     

Statistical monitoring of nonlinear profiles by using piecewise linear approximation

In many practical situations, the quality of a process, or product, is better characterized and summarized by the relationship between a response variable and one or more explanatory variables. Such a relationship between the response variable and explanatory variables is called a profile. Recently, profile monitoring has become a fertile research field in statistical process control (SPC). To handle the nonlinear profile data, the proposal considered in this paper is that the entire curve is broken into several segments of data points that exhibit a statistical fit to the linear model, and therefore each of them can be monitored separately by using existing linear profile SPC methods. A new method that determines the locations of change points based on the slop change is proposed. Two goodness-of-fit criteria are utilized for determining the best number of change points to avoid over-fitting. Two nonlinear profile examples taken from the literature are used to illustrate the proposed change-point model. Monitoring performances using the existing T² and EWMA-based approaches are presented when the nonlinear profile data is fitted by using the proposed change-point model.     

Fields of non-linear regression models for atmospheric correction of satellite ocean-color imagery

Remote sensing of ocean color from space, a problem that consists of retrieving spectral marine reflectance from spectral top-of-atmosphere reflectance, is considered as a collection of similar inverse problems continuously indexed by the angular variables influencing the observation process. A general solution is proposed in the form of a field of non-linear regression models over the set T of permitted values for the angular variables, i.e., as a map from T to some function space. Each value of the field is a regression model that performs a direct mapping from the top-of-atmosphere reflectance to the marine reflectance. Since the spectral components of the field take values in the same variable vector space, the retrievals in individual spectral bands are not independent, i.e., the solution is not just a juxtaposition of independent models for each spectral band. A scheme based on ridge functions is developed to approximate this solution to an arbitrary accuracy, and is applied to the retrieval of marine reflectance in Case 1 waters, for which optical properties are only governed by biogenic content. The statistical models are evaluated on synthetic data as well as actual data originating from the SeaWiFS instrument, taking into account noise in the data. Theoretical performance is good in terms of accuracy, robustness, and generalization capabilities, suggesting that the function field methodology might improve atmospheric correction in the presence of absorbing aerosols and provide more accurate estimates of marine reflectance in productive waters. When applied to SeaWiFS imagery acquired off California, the function field methodology gives generally higher estimates of marine reflectance than the standard SeaDAS algorithm, but the values are more realistic.     

Improved fault detection employing hybrid memetic fuzzy modeling and adaptive filters

We propose an improved fault detection (FD) scheme based on residual signals extracted on-line from system models identified from high-dimensional measurement data recorded in multi-sensor networks. The system models are designed for an all-coverage approach and comprise linear and non-linear approximation functions representing the interrelations and dependencies among the measurement variables. The residuals obtained by comparing observed versus predicted values (i.e., the predictions achieved by the system models) are normalized subject to the uncertainty of the models and are supervised by an incrementally adaptive statistical tolerance band. Upon violation of this tolerance band, a fault alarm is triggered. The improved FD methods comes with two the main novelty aspects: (1) the development of an enhanced optimization scheme for fuzzy systems training which builds upon the SparseFIS (Sparse Fuzzy Inference Systems) approach and enhances it by embedding genetic operators for escaping local minima ⟶ a hybrid memetic (sparse) fuzzy modeling approach, termed as GenSparseFIS. (2) The design and application of adaptive filters on the residual signals, over time, in a sliding-window based incremental/decremental manner to smoothen the signals and to reduce the false positive rates. This gives us the freedom to tighten the tolerance band and thus to increase fault detection rates by holding the same level of false positives. In the results section, we verify that this increase is statistically significant in the case of adaptive filters when applying the proposed concepts onto four real-world scenarios (three different ones from rolling mills, one from engine test benches). The hybridization of sparse fuzzy inference systems with genetic algorithms led to the generation of more high quality models that can in turn be used in the FD process as residual generators. The new hybrid sparse memetic modeling approach also achieved fuzzy systems leading to higher fault detection rates for some scenarios.     

Subgrid modeling for convection-diffusion-reaction in one space dimension using a Haar Multiresolution analysis

In this paper we propose and study a subgrid model for linear convection-diffusion-reaction equations with fractal rough coefficients. The subgrid model is based on scale extrapolation of a modeling residual from coarser scales using a computed solution on a finest scale as reference. We show in experiments that a solution with subgrid model on a scale h in most cases corresponds to a solution without subgrid model on a scale less than h/4. We also present error estimates for the modeling error in terms of modeling residuals.     

Multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for OCR prediction

This article investigates the feasibility of multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for the prediction of over consolidation ratio (OCR) of clay deposits based on Piezocone Penetration Tests (PCPT) data. MARS uses piece-wise linear segments to describe the non-linear relationships between input and output variables. LSSVM is firmly based on the theory of statistical learning, and uses regression technique. The input parameters of the models are corrected cone resistance (q _t ), vertical total stress (σ_v), hydrostatic pore pressure (u ₀), pore pressure at the cone tip (u ₁), and the pore pressure just above the cone base (u ₂). The developed LSSVM model gives error bar of predicted OCR. Equations have also been developed for prediction of OCR. The performance of MARS and LSSVM models has been compared with the traditional methods for OCR prediction. As the results reveal, the proposed MARS and LSSVM models are robust models for determination of OCR.     

Signature extraction using mutual interdependencies

Recently, mutual interdependence analysis (MIA) has been successfully used to extract representations, or “mutual features”, accounting for samples in the class. For example, a mutual feature is a face signature under varying illumination conditions or a speaker signature under varying channel conditions. A mutual feature is a linear regression that is equally correlated with all samples of the input class. Previous work discussed two equivalent definitions of this problem and a generalization of its solution called generalized MIA (GMIA). Moreover, it showed how mutual features can be computed and employed. This paper uses a parametrized version GMIA(λ) to pursue a deeper understanding of what GMIA features really represent. It defines a generative signal model that is used to interpret GMIA(λ) and visualize its difference to MIA, principal and independent component analysis. Finally, we analyze the effect of λ on the feature extraction performance of GMIA(λ) in two standard pattern recognition problems: illumination-independent face recognition and text-independent speaker verification.     

A Bayes-true data generator for evaluation of supervised and unsupervised learning methods

Benchmarking pattern recognition, machine learning and data mining methods commonly relies on real-world data sets. However, there are some disadvantages in using real-world data. On one hand collecting real-world data can become difficult or impossible for various reasons, on the other hand real-world variables are hard to control, even in the problem domain; in the feature domain, where most statistical learning methods operate, exercising control is even more difficult and hence rarely attempted. This is at odds with the scientific experimentation guidelines mandating the use of as directly controllable and as directly observable variables as possible. Because of this, synthetic data possesses certain advantages over real-world data sets. In this paper we propose a method that produces synthetic data with guaranteed global and class-specific statistical properties. This method is based on overlapping class densities placed on the corners of a regular k-simplex. This generator can be used for algorithm testing and fair performance evaluation of statistical learning methods. Because of the strong properties of this generator researchers can reproduce each others experiments by knowing the parameters used, instead of transmitting large data sets.     

Natural facial expression recognition using differential-AAM and manifold learning

This paper proposes a novel natural facial expression recognition method that recognizes a sequence of dynamic facial expression images using the differential active appearance model (AAM) and manifold learning as follows. First, the differential-AAM features (DAFs) are computed by the difference of the AAM parameters between an input face image and a reference (neutral expression) face image. Second, manifold learning embeds the DAFs on the smooth and continuous feature space. Third, the input facial expression is recognized through two steps: (1) computing the distances between the input image sequence and gallery image sequences using directed Hausdorff distance (DHD) and (2) selecting the expression by a majority voting of k-nearest neighbors (k-NN) sequences in the gallery. The DAFs are robust and efficient for the facial expression analysis due to the elimination of the inter-person, camera, and illumination variations. Since the DAFs treat the neutral expression image as the reference image, the neutral expression image must be found effectively. This is done via the differential facial expression probability density model (DFEPDM) using the kernel density approximation of the positively directional DAFs changing from neutral to angry (happy, surprised) and negatively directional DAFs changing from angry (happy, surprised) to neutral. Then, a face image is considered to be the neutral expression if it has the maximum DFEPDM in the input sequences. Experimental results show that (1) the DAFs improve the facial expression recognition performance over conventional AAM features by 20% and (2) the sequence-based k-NN classifier provides a 95% facial expression recognition performance on the facial expression database (FED06).     

ENTROPY MINIMAX MULTIVARIATE STATISTICAL MODELING—II: APPLICATIONS

Applications of entropy minimax are summarized in three major areas: meteorology, engineering/ materials science, and medicine/biology. The applications cover both discrete patterns in multidimensional spaces of mixed quantitative and qualitative variables, and continuous patterns employing concepts of potential functions and fuzzy entropies. Major achievements of entropy minimax modeling include the first long range weather forecasting models with statistical reliability significantly above chance verified on independent data, the first models of fission gas release and nuclear fuel failure under commercial operating conditions with significant and independently verified statistical reliability, and the first prognosis models in coronary artery disease and in non-Hodgkin's lymphoma with significant predictability verified on independent data. In addition, applications of entropy minimization and maximization separately are reviewed, including feature selection, unsupervised classification, probability estimation, statistical distribution determination, statistical mechanics and thermodynamics, pattern recognition, spectral analysis and image reconstruction. Comparisons between entropy minimax and other methodolodies are provided, including sample average predictors, nearest neighbors predictors, linear regression, logistic regression, Cox proportional hazards regression, recursive partitioning, linear discriminant analysis, mechanistic modeling, and expert (heuristic) programming.