A novel evolutionary-negative correlated mixture of experts model in tourism demand estimation

Mixtures of experts (ME) model are widely used in many different areas as a recognized ensemble learning approach to account for nonlinearities and other complexities in the data, such as time series estimation. With the aim of developing an accurate tourism demand time series estimation model, a mixture of experts model called LSPME (Lag Space Projected ME) is presented by combining ideas from subspace projection methods and negative correlation learning (NCL). The LSPME uses a new cluster-based lag space projection (CLSP) method to automatically obtain input space to train each expert focused on the difficult instances at each step of the boosting approach. For training experts of the LSPME, a new NCL algorithm called Sequential Evolutionary NCL algorithm (SENCL) is proposed that uses a moving average for the correlation penalty term in the error function of each expert to measure the error correlation between it and its previous experts. The LSPME model was compared with other ensemble models using monthly tourist arrivals to Japan from four markets: The United States, United Kingdom, Hong Kong and Taiwan. The experimental results show that the estimation accuracy of the proposed LSPME model is significantly better than the other ensemble models and can be considered to be a promising alternative for time series estimation problems.     

Large vocabulary word recognition using context-dependent allophonic hidden Markov models

In this paper, we report our development of context-dependent allophonic hidden Markov models (HMMs) implemented in a 75 000-word speaker-dependent Gaussian-HMM recognizer. The context explored is the immediate left and/or right adjacent phoneme. To achieve reliable estimation of the model parameters, phonemes are grouped into classes based on their expected co-articulatory effects on neighboring phonemes. Only five separate preceding and following contexts are identified explicitly for each phoneme. By grouping the contexts we ensure that they occur frequently enough in the training data to allow reliable estimation of the parameters of the HMM representing the context-dependent units. Further improvement in the estimation reliability is obtained by tying the covariance matrices in the HMM output distributions across all contexts. Speech recognition experiments show that when a large amount of data (e.g. over 2500 words) is used to train context-dependent HMMs, the word recognition error rate is reduced by 33%, compared with the context-independent HMMs. For smaller amounts of training data the error reduction becomes less significant.     

Dynamic maintenance of data distribution for selectivity estimation

We propose a new dynamic method for multidimensional selectivity estimation for range queries that works accurately independent of data distribution. Good estimation of selectivity is important for query optimization and physical database design. Our method employs the multilevel grid file (MLGF) for accurate estimation of multidimensional data distribution. The MLGF is a dynamic, hierarchical, balanced, multidimensional file structure that gracefully adapts to nonuniform and correlated distributions. We show that the MLGF directory naturally represents a multidimensional data distribution. We then extend it for further refinement and present the selectivity estimation method based on the MLGF. Extensive experiments have been performed to test the accuracy of selectivity estimation. The results show that estimation errors are very small independent of distributions, even with correlated and/or highly skewed ones. Finally, we analyze the cause of errors in estimation and investigate the effects of various parameters on the accuracy of estimation.     

Transform both sides model: A parametric approach

A parametric regression model for right-censored data with a log-linear median regression function and a transformation in both response and regression parts, named parametric Transform-Both-Sides (TBS) model, is presented. The TBS model has a parameter that handles data asymmetry while allowing various different distributions for the error, as long as they are unimodal symmetric distributions centered at zero. The discussion is focused on the estimation procedure with five important error distributions (normal, double-exponential, Student’s t, Cauchy and logistic) and presents properties, associated functions (that is, survival and hazard functions) and estimation methods based on maximum likelihood and on the Bayesian paradigm. These procedures are implemented in TBSSurvival, an open-source fully documented R package. The use of the package is illustrated and the performance of the model is analyzed using both simulated and real data sets.     

Measurement error in statistical models of shape

Active shape models (ASMs) are popular and sophisticated methods of extracting features in (especially medical) images. Here we analyse the error in placing ASM points on the boundary of the feature. By using replications, a corrected covariance matrix is presented that should reduce the effects of placement error. We show analytically and via simulations that the cumulative variability for a given number of eigenvalues retained in principal components analysis (PCA) ought to be reduced by increasing levels of point-placement error. Results for predicted errors are in excellent agreement with the set-up parameters of two simulated shapes and with anecdotal evidence from the trained experts for real data taken from the OSTEODENT project. We derive an equation for the reliability of placing the points and we find values of 0.79 and 0.85 (where 0 = bad and 1 = good) for the two clinical experts for the OSTEODENT data. These analyses help us to understand the sources and effects of measurement error in shape models.     

A quantile estimation for massive data with generalized Pareto distribution

This paper proposes a new method of estimating extreme quantiles of heavy-tailed distributions for massive data. The method utilizes the Peak Over Threshold (POT) method with generalized Pareto distribution (GPD) that is commonly used to estimate extreme quantiles and the parameter estimation of GPD using the empirical distribution function (EDF) and nonlinear least squares (NLS). We first estimate the parameters of GPD using EDF and NLS and then, estimate multiple high quantiles for massive data based on observations over a certain threshold value using the conventional POT. The simulation results demonstrate that our parameter estimation method has a smaller Mean square error (MSE) than other common methods when the shape parameter of GPD is at least 0. The estimated quantiles also show the best performance in terms of root MSE (RMSE) and absolute relative bias (ARB) for heavy-tailed distributions.     

Optimal approximation of signal priors

In signal restoration by Bayesian inference, one typically uses a parametric model of the prior distribution of the signal. Here, we consider how the parameters of a prior model should be estimated from observations of uncorrupted signals. A lot of recent work has implicitly assumed that maximum likelihood estimation is the optimal estimation method. Our results imply that this is not the case. We first obtain an objective function that approximates the error occurred in signal restoration due to an imperfect prior model. Next, we show that in an important special case (small gaussian noise), the error is the same as the score-matching objective function, which was previously proposed as an alternative for likelihood based on purely computational considerations. Our analysis thus shows that score matching combines computational simplicity with statistical optimality in signal restoration, providing a viable alternative to maximum likelihood methods. We also show how the method leads to a new intuitive and geometric interpretation of structure inherent in probability distributions.     

Development of a fully automated RULA assessment system based on computer vision

The purpose of this study was to develop an automated, RULA-based posture assessment system using a deep learning algorithm to estimate RULA scores, including scores for wrist posture, based on images of workplace postures. The proposed posture estimation system reported a mean absolute error (MAE) of 2.86 on the validation dataset obtained by randomly splitting 20% of the original training dataset before data augmentation. The results of the proposed system were compared with those of two experts’ manual evaluation by computing the intraclass correlation coefficient (ICC), which yielded index values greater than 0.75, thereby confirming good agreement between manual raters and the proposed system. This system will reduce the time required for postural evaluation while producing highly reliable RULA scores that are consistent with those generated by manual approach. Thus, we expect that this study will aid ergonomic experts in conducting RULA-based surveys of occupational postures in workplace conditions.     

Improving biosignal processing through modeling uncertainty: Bayes vs. non-bayes in sleep staging

In this paper we report about an investigation of Bayesian inference applied to neural networks multilayer perceptrons (MLP), in particular in the task of automatic sleep staging based on electroencephalogram (EEG) and electrooculogram (EOG) signals. The main focus was on evaluating the use of so-called "doubt-levels" and "confidence intervals" ("error bars") in improving the results by rejecting uncertain cases and patterns not well represented by the training set. Bayesian inference is used to arrive at distributions of network weights based on training data. We compare the results of the full-blown Bayesian method with results obtained from a k-nearest neighbor classifier. The results show that the Bayesian technique significantly outperforms the k-nearest-neighbor classifier. At the same time, we show that Bayesian inference, for which we have developed an extension for the calculation of error bars in the latent space of hidden units, can indeed be used for improving results by rejecting cases below a doubt-level threshold of probability, as well as for the rejection of artifacts. The performance of the Bayesian solution, however, is not significantly better than alternative techniques such as doubt levels applied to a maximum posterior approach, or the use of density estimation for outlier rejection. We conclude that Bayesian inference is a valid and valuable technique for model estimation but in the given application does not lead to improved results over simpler techniques.     

观测误差协方差估计下的集合鲁棒滤波数据同化方法

在数据同化方法中,观测误差协方差矩阵是相关的,且与时间和状态有一定的依赖性.针对这种相关特性,将鲁棒滤波方法与观测误差协方差估计方法相结合,得到随状态时间变化的观测误差协方差,提出一种带有观测误差估计的鲁棒数据同化新方法,更新观测误差协方差,改善估计效果.从分析误差协方差,转移矩阵特征值放大等角度优化同化方法.利用非线...     

基于序贯最小二乘的多传感器误差配准方法

为实时估计多传感器系统偏差,针对广义最小二乘(GLS)配准方法不能实时估计传感器偏差的问题,提出了基于序贯最小二乘的多传感器误差估计方法,该方法在GLS配准模型基础上,采用最小二乘的序贯方法来估计系统偏差,不必存储过去的测量数据,能够实时估计系统偏差。仿真结果表明了该方法的有效性。     

On the performance of the flexible maximum entropy distributions within partially adaptive estimation

The partially adaptive estimation based on the assumed error distribution has emerged as a popular approach for estimating a regression model with non-normal errors. In this approach, if the assumed distribution is flexible enough to accommodate the shape of the true underlying error distribution, the efficiency of the partially adaptive estimator is expected to be close to the efficiency of the maximum likelihood estimator based on knowledge of the true error distribution. In this context, the maximum entropy distributions have attracted interest since such distributions have a very flexible functional form and nest most of the statistical distributions. Therefore, several flexible MaxEnt distributions under certain moment constraints are determined to use within the partially adaptive estimation procedure and their performances are evaluated relative to well-known estimators. The simulation results indicate that the determined partially adaptive estimators perform well for non-normal error distributions. In particular, some can be useful in dealing with small sample sizes. In addition, various linear regression applications with non-normal errors are provided.     

基于神经网络的一类生化反应器状态在线估计

提出一种随机非线性系统状态和参数同时估计的神经网络新方法，并证明了该方法的无偏性和是小方差性，将其用于乙醇间歇发酵器的状态和参数估计，结果表明估计值民实验值相吻合，此方法对噪声特片无特殊要求，对初始状态估值不敏感，对初始参数值具有一定的鲁棒性，可利用有限的状态量测信息在线估计不可测量的状态变量和物理参数。     

Transfer estimation of evolving class priors in data stream classification

Data stream classification is a hot topic in data mining research. The great challenge is that the class priors may evolve along the data sequence. Algorithms have been proposed to estimate the dynamic class priors and adjust the classifier accordingly. However, the existing algorithms do not perform well on prior estimation due to the lack of samples from the target distribution. Sample size has great effects in parameter estimation and small-sample effects greatly contaminate the estimation performance. In this paper, we propose a novel parameter estimation method called transfer estimation. Transfer estimation makes use of samples not only from the target distribution but also from similar distributions. We apply this new estimation method to the existing algorithms and obtain an improved algorithm. Experiments on both synthetic and real data sets show that the improved algorithm outperforms the existing algorithms on both class prior estimation and classification.     

基于动态概率抽样的标签噪声过滤方法

在机器学习问题中,数据质量对系统预测的准确性产生了深远的影响。由于信息获取的难度大,人类的认知主观且有限,导致了专家无法准确标记所有样本。而近年来出现的一些概率抽样方法无法避免样本人为划分不合理且主观性较强的问题。针对这一问题,提出一种基于动态概率抽样（DPS）的标签噪声过滤方法,充分考虑各个数据集样本间的差异性,通过统计各个区间内置信度分布频率,分析各个区间内置信度分布信息熵的走势,确定合理阈值。在UCI经典数据集中选取了14个数据集,将所提方法与随机森林（RF）、HARF、MVF、局部概率抽样（LPS）等方法进行了对比实验。实验结果表明,所提出的方法在标签噪声识别和分类泛化上均展示出了较高的能力。     

Moments and root-mean-square error of the Bayesian MMSE estimator of classification error in the Gaussian model

The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been designed. Use of prior knowledge, in the form of a prior distribution on an uncertainty class of feature-label distributions to which the true, but unknown, feature-distribution belongs, can facilitate accurate error estimation (in the mean-square sense) in circumstances where accurate completely model-free error estimation is impossible. This paper provides analytic asymptotically exact finite-sample approximations for various performance metrics of the resulting Bayesian Minimum Mean-Square-Error (MMSE) error estimator in the case of linear discriminant analysis (LDA) in the multivariate Gaussian model. These performance metrics include the first, second, and cross moments of the Bayesian MMSE error estimator with the true error of LDA, and therefore, the root-mean-square (RMS) error of the estimator. We lay down the theoretical groundwork for Kolmogorov double-asymptotics in a Bayesian setting, which enables us to derive asymptotic expressions of the desired performance metrics. From these we produce analytic finite-sample approximations and demonstrate their accuracy via numerical examples. Various examples illustrate the behavior of these approximations and their use in determining the necessary sample size to achieve a desired RMS. The Supplementary Material contains derivations for some equations and added figures.     

ROBUSTNESS INSTEAD OF ACCURACY SHOULD BE THE PRIMARY OBJECTIVE FOR SUBJECTIVE PATTERN RECOGNITION RESEARCH: STABILITY ANALYSIS ON MULTICANDIDATE ELECTORAL COLLEGE VERSUS DIRECT POPULAR VOTE

Subjective pattern recognition is a class of pattern recognition problems, where we not only merely know a few, if any, the strategies our brains employ in making decisions in daily life but also have only limited ideas on the standards our brains use in determining the equality/inequality among the objects. Face recognition is a typical example of such problems. For solving a subjective pattern recognition problem by machinery, application accuracy is the standard performance metric for evaluating algorithms. However, we indeed do not know the connection between algorithm design and application accuracy in subjective pattern recognition. Consequently, the research in this area follows a “trial and error” process in a general sense: try different parameters of an algorithm, try different algorithms, and try different algorithms with different parameters. This phenomenon can be observed clearly in the nearly 30 years research of the face recognition: although huge advances have been made, no algorithm has ever been shown a potential to be consistently better than most of the algorithms developed earlier; it was even shown that a naïve algorithm can work, in the sense of accuracy, at least no worse than many newly developed ones in a few benchmarks. We argue that, the primary objective of subjective pattern recognition research should be moved to theoretical robustness from application accuracy so that we can evaluate and compare algorithms without or with only few “trial and error” steps. We in this paper introduce an analytical model for studying the theoretical stabilities of multicandidate Electoral College and Direct Popular Vote schemes (aka regional voting scheme and national voting scheme, respectively), which can be expressed as the a posteriori probability that a winning candidate will continue to be chosen after the system is subjected to noise. This model shows that, in the context of multicandidate elections, generally, Electoral College is more stable than Direct Popular Vote, that the stability of Electoral College increases from that of Direct Popular Vote as the size of the subdivided regions decreases from the original nation size, up to a certain level, and then the stability starts to decrease approaching the stability of Direct Popular Vote as the region size approaches the original unit cell size; and that the stability of Electoral College approaches that of Direct Popular Vote in the two extremities as the region size increases to the original national size or decreases to the unit cell size. It also shows a special situation of white noise dominance with negligibly small concentrated noise, where Direct Popular Vote is surprisingly more stable than Electoral College, although the existence of such a special situation is questionable. We observe that “high stability” in theory indeed always reveals itself in “high accuracy” in applications. Extensive experiments on two human face benchmark databases applying an Electoral College framework embedded with standard baseline and newly developed holistic algorithms have been conducted. The impressive improvement by Electoral College over regular holistic algorithms verifies the stability theory on the voting systems. It also shows an evidential support for adopting theoretical stability instead of application accuracy as the primary objective for subjective pattern recognition research.     

Sentiment Analysis with Tweets Behaviour in Twitter Streaming API

Twitter is a radiant platform with a quick and effective technique to analyze users’ perceptions of activities on social media. Many researchers and industry experts show their attention to Twitter sentiment analysis to recognize the stakeholder group. The sentiment analysis needs an advanced level of approaches including adoption to encompass data sentiment analysis and various machine learning tools. An assessment of sentiment analysis in multiple fields that affect their elevations among the people in real-time by using Naive Bayes and Support Vector Machine (SVM). This paper focused on analysing the distinguished sentiment techniques in tweets behaviour datasets for various spheres such as healthcare, behaviour estimation, etc. In addition, the results in this work explore and validate the statistical machine learning classifiers that provide the accuracy percentages attained in terms of positive, negative and neutral tweets. In this work, we obligated Twitter Application Programming Interface (API) account and programmed in python for sentiment analysis approach for the computational measure of user’s perceptions that extract a massive number of tweets and provide market value to the Twitter account proprietor. To distinguish the results in terms of the performance evaluation, an error analysis investigates the features of various stakeholders comprising social media analytics researchers, Natural Language Processing (NLP) developers, engineering managers and experts involved to have a decision-making approach.     

Residual variance estimation in machine learning

The problem of residual variance estimation consists of estimating the best possible generalization error obtainable by any model based on a finite sample of data. Even though it is a natural generalization of linear correlation, residual variance estimation in its general form has attracted relatively little attention in machine learning.In this paper, we examine four different residual variance estimators and analyze their properties both theoretically and experimentally to understand better their applicability in machine learning problems. The theoretical treatment differs from previous work by being based on a general formulation of the problem covering also heteroscedastic noise in contrary to previous work, which concentrates on homoscedastic and additive noise.In the second part of the paper, we demonstrate practical applications in input and model structure selection. The experimental results show that using residual variance estimators in these tasks gives good results often with a reduced computational complexity, while the nearest neighbor estimators are simple and easy to implement.