Clustering with block mixture models

Basing cluster analysis on mixture models has become a classical and powerful approach. Until now, this approach, which allows to explain some classic clustering criteria such as the well-known k-means criteria and to propose general criteria, has been developed to classify a set of objects measured on a set of variables. But, for this kind of data, if most clustering procedures are designated to construct an optimal partition of objects or, sometimes, of variables, there exist others methods, named block clustering methods, which consider simultaneously the two sets and organize the data into homogeneous blocks.In this work, a new mixture model called block mixture model is proposed to take into account this situation. This model allows to embed simultaneous clustering of objects and variables in a mixture approach. We first consider this probabilistic model in a general context and we develop a new algorithm of simultaneous partitioning based on the CEM algorithm. Then, we focus on the case of binary data and we show that our approach allows us to extend a block clustering method, which had been proposed in this case. Simplicity, fast convergence and the possibility to process large data sets are the major advantages of the proposed approach.     

Constrained predictive pole-placement control with linear models

Predictive pole-placement (PPP) control is a continuous-time MPC using a particular set of basis functions leading to pole-placement behaviour in the unconstrained case. This paper presents two modified versions of the PPP controller which are each shown to have desirable stability properties when controlling systems with input, output and state constraints.     

A stable block model predictive control with variable implementation horizon

In this paper, we present a stable model predictive control method for discrete-time nonlinear systems. The standard MPC scheme is modified to incorporate (1) a block implementation scheme where a sub-string of the optimized input sequence is applied instead of a single value; (2) an additional constraint which guarantees that a Lyapunov function will decrease over time; (3) a variable implementation window that facilitates the stability constraint enforcement. Stability of the closed-loop system with the proposed algorithm is established. Examples are given to illustrate the effectiveness of the control scheme. The impacts of several key design parameters on the overall performance are also analyzed and discussed.     

An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models

An important objective of data mining is the development of predictive models. Based on a number of observations, a model is constructed that allows the analysts to provide classifications or predictions for new observations. Currently, most research focuses on improving the accuracy or precision of these models and comparatively little research has been undertaken to increase their comprehensibility to the analyst or end-user. This is mainly due to the subjective nature of ‘comprehensibility’, which depends on many factors outside the model, such as the user's experience and his/her prior knowledge. Despite this influence of the observer, some representation formats are generally considered to be more easily interpretable than others. In this paper, an empirical study is presented which investigates the suitability of a number of alternative representation formats for classification when interpretability is a key requirement. The formats under consideration are decision tables, (binary) decision trees, propositional rules, and oblique rules. An end-user experiment was designed to test the accuracy, response time, and answer confidence for a set of problem-solving tasks involving the former representations. Analysis of the results reveals that decision tables perform significantly better on all three criteria, while post-test voting also reveals a clear preference of users for decision tables in terms of ease of use.     

Model predictive control for LPV models with maximal stabilizable model range

This paper characterizes model predictive control (MPC) for linear parameter varying (LPV) models subject to state and input constraints, which is based on the homogeneous polynomially parameterized (HPP) Lyapunov function and HPP control law with tunable complexity degrees. The controller guarantees the closed‐loop asymptotic stability and finds the control move through the convex optimization. While it is known that this technique can improve the control performance and reduce conservatism, we suggest that it also enlarges, and maximizes with the sufficiently large complexity degrees, the stabilizable LPV model range. The computational burden becomes heavier when the complexity degrees increase. However, the main contributions of this paper are more on theory than on practice. It explores to what extent robust MPC can be applied for stabilization of LPV models. Numerical examples are provided to illustrate the effectiveness of the proposed technique.     

热物理性质预测模型的合理性分析-模型的bias问题

化学工业及科学研究极需化合物热力学性质的预测模型.但是现有的大多数模型可靠性很低.这主要是由于可用于发展模型的试验数据往往太少,以至于所用到的数据经常缺乏代表性,使模型在作预测时易发生大的误差,甚至严重的错误.因此,如果数据的代表性问题不解决,则无论任何数学模型,优化方法,神经网络或进化算法都无法真正改进模型的预测能力,为了全面地理解模型的预测能力,奉文建议除了要对模型作基于试验数据的检验,还应对模型作基于规则的分析-即模型的合理性分析.该分析强调用各种结构类型的化合物对模型预测值的合理性作基于热力学原理和与已知试验数槲倾向一致性的检验.这种分析方法不仅有助于全面地了解一个模型的预测能力,而且有助于发展可靠的模型.     

The management and mining of multiple predictive models using the predictive modeling markup language

We introduce a markup language based upon XML for working with the predictive models produced by data mining systems. The language is called the predictive model markup language (PMML) and can be used to define predictive models and ensembles of predictive models. It provides a flexible mechanism for defining schema for predictive models and supports model selection and model averaging, involving multiple predictive models. It has proved useful for applications requiring ensemble learning, partitioned learning and distributed learning. In addition, it facilitates moving predictive models across applications and systems.     

Perception of human motion with different geometric models

Human figures have been animated using a variety of geometric models, including stick figures, polygonal models and NURBS-based models with muscles, flexible skin or clothing. This paper reports on experimental results indicating that a viewer's perception of motion characteristics is affected by the geometric model used for rendering. Subjects were shown a series of paired motion sequences and asked if the two motions in each pair were the same or different. The motion sequences in each pair were rendered using the same geometric model. For the three types of motion variation tested, sensitivity scores indicate that subjects were better able to observe changes with the polygonal model than they were with the stick-figure model     

Choice of generalized linear mixed models using predictive crossvalidation

The choice of generalized linear mixed models is difficult, because it involves the selection of both fixed and random effects. Classical criteria like Akaike’s information criterion (AIC) are often not suitable for the latter task, and others which are useful in linear mixed models are difficult to extend to the generalized case, especially for overdispersed data. A predictive leave-one-out crossvalidation approach is suggested that can be applied for choosing both fixed and random effects, even in models with overdispersion, and is based on proper scoring rules. An attractive feature of this approach is the fact that the model has to be fitted just once to the data set, which makes computations fast and convenient. As the calculation of the leave-one-out predictive distribution is not possible analytically, it is shown how an iteratively weighted least squares algorithm combined with some analytic approximations can be used for this task. A simulation study and two applications of the methodology to binary and count data are provided, as well as comparisons with two other methods.     

Camera calibration with distortion models and accuracy evaluation

A camera model that accounts for major sources of camera distortion, namely, radial, decentering, and thin prism distortions is presented. The proposed calibration procedure consists of two steps: (1) the calibration parameters are estimated using a closed-form solution based on a distribution-free camera model; and (2) the parameters estimated in the first step are improved iteratively through a nonlinear optimization, taking into account camera distortions. According to minimum variance estimation, the objective function to be minimized is the mean-square discrepancy between the observed image points and their inferred image projections computed with the estimated calibration parameters. The authors introduce a type of measure that can be used to directly evaluate the performance of calibration and compare calibrations among different systems. The validity and performance of the calibration procedure are tested with both synthetic data and real images taken by tele- and wide-angle lenses     

双不确定相依竞争失效模型的可靠性评估

复杂设备常会经历自然磨损和外力影响,外力致使磨损量突然增大.在工程实践中,很多高可靠性、长寿命的设备,由于不易获得足量的失效数据,须使用专家的经验数据来评估设备的寿命分布和可靠性指标.然而,专家的经验数据往往不是精确的常数,而是一个限定范围.为了处理这种人为认知引起的不确定性,引入不确定理论,用不确定变量刻画专家的经验...     

Discovery and learning of models with predictive state representations for dynamical systems without reset

Modeling dynamical systems is a common problem in science and engineering. After a system has been modeled, the system can be controlled and predicted. Predictive state representations (PSRs) is a recently proposed method of modeling controlled dynamical systems. One central problem in the PSRs literature is concerned with discovery and learning of PSRs. This paper presents a new algorithm for discovery and learning of PSRs by using only a continuous trace of actions and observations as the training data, in which the history at any time step in the training data can be identified, and then the prediction of test at a history and the PSR model of the system can be obtained. We empirically evaluate and compare our algorithm on a standard set of POMDP test problems and the empirical results show that our algorithm is competitive and outperforms the suffix-history algorithm.     

Nonlinear model predictive control of multivariable processes using block-structured models

Block-structured models, such as Wiener or Hammerstein models, have been used in nonlinear model predictive control to reduce the cost of identification and online computation. The solution of a nonlinear dynamic optimization problem has been avoided by inverting the nonlinear element and solving the resulting linear problem in the past. However, by exploiting the block structure for sensitivity calculation, the original nonlinear problem can also be solved at low computational cost. At the same time, greater modeling flexibility is achieved. Recently, a new Hammerstein model structure has been proposed for multivariable processes with input directionality, which exploits such increased modeling flexibility. This paper deals with nonlinear model predictive control constrained by models of Hammerstein or Uryson structure. A method for efficient calculation of sensitivity information is developed. In a simulation example, the method is shown to combine low computational cost with a significant reduction of the loss of optimality compared to the previous methods.     

Ranking-based evaluation of regression models

We suggest the use of ranking-based evaluation measures for regression models, as a complement to the commonly used residual-based evaluation. We argue that in some cases, such as the case study we present, ranking can be the main underlying goal in building a regression model, and ranking performance is the correct evaluation metric. However, even when ranking is not the contextually correct performance metric, the measures we explore still have significant advantages: They are robust against extreme outliers in the evaluation set; and they are interpretable. The two measures we consider correspond closely to non-parametric correlation coefficients commonly used in data analysis (Spearman's ρ and Kendall's τ); and they both have interesting graphical representations, which, similarly to ROC curves, offer useful various model performance views, in addition to a one-number summary in the area under the curve. An interesting extension which we explore is to evaluate models on their performance in “partially” ranking the data, which we argue can better represent the utility of the model in many cases. We illustrate our methods on a case study of evaluating IT Wallet size estimation models for IBM's customers. Saharon Rosset is Research Staff Member in the Data Analytics Research Group at IBM's T. J. Watson Research Center. He received his B.S. in Mathematics and M.Sc., in Statistics from Tel Aviv University in Israel, and his Ph.D. in Statistics from Stanford University in 2003. In his research, he aspires to develop practically useful predictive modeling methodologies and tools, and apply them to solve problems in business and scientific domains. Currently, his major projects include work on customer wallet estimation and analysis of genetic data. Claudia Perlich has received a M.Sc. in Computer Science from Colorado University at Boulder, a Diploma in Computer Science from Technische Universitaet in Darmstadt, and her Ph.D. in Information Systems from Stern School of Business, New York University. Her Ph.D. thesis concentrated on probability estimation in multi-relational domains that capture information of multiple entity types and relationships between them. Her dissertation was recognized as an additional winner of the International SAP Doctoral Support Award Competition. Claudia joined the Data Analytics Research group at IBM's T.J. Watson Research Center as a Research Staff Member in October 2004. Her research interests are in statistical machine learning for complex real-world domains and business applications. Bianca Zadrozny is currently an associate professor at the Computer Science Department of Federal Fluminense University in Brazil. Her research interests are in the areas of applied machine learning and data mining. She received her B.Sc. in Computer Engineering from the Pontifical Catholic University in Rio de Janeiro, Brazil, and her M.Sc. and Ph.D. in Computer Science from the University of California at San Diego. She has also worked as a research staff member in the data analytics research group at IBM T.J. Watson Research Center.     

Generalized predictive control and tuning of industrial processes with second order plus dead time models

In this paper, an extension of the modified generalized predictive control (GPC) algorithm and a tuning strategy is presented. To take the plant dynamics such as under damped behavior and the effect of zeros into account, extension to the second order plus dead time (SOPDT) of the first order plus dead time (FOPDT) modified GPC method is proposed. It is shown that this method is computationally undemanding. Also, implementation is more straightforward than conventional GPC algorithms. Moreover, the proposed tuning strategy enables a fast implementation of the GPC with regard to nominal stability and desired performance. The simplicity of this strategy and its wide applicability makes it readily accessible to practitioners for utilization. Multiple simulation results are provided to show the effectiveness of the proposed algorithm.     

Parameter bounds evaluation of Wiener models with noninvertible polynomial nonlinearities

In this paper a three stage procedure is presented for deriving parameters bounds of SISO Wiener models when the nonlinear block is modeled by a possibly noninvertible polynomial and the output measurement errors are bounded. First, using steady-state input-output data, parameters of the nonlinear part are bounded by a tight orthotope. Then, given the estimated uncertain nonlinearity and the output measurements collected exciting the system with an input dynamic signal, bounds on the unmeasurable inner signal are computed. Finally, such bounds, together with noisy output measurements, are used for bounding the parameters of the linear block.     

Managing diagrammatic models with different perspectives on product information

In the context of engineering-oriented companies, a basic distinction can be made between the information needed during sales, engineering and manufacturing. When representing such information in diagrammatic models, these models will most often include both individual and shared content. For instance, during the sales phase, information about sales prices may be relevant, while this information is not relevant in the engineering and manufacturing phases, where, on the other hand, more detailed information about components and assembly is needed. However, such information models often share basic component definitions. Having this overlapping information across models means that when maintaining these, redundant work has to be carried out for the overlapping parts of the information. This, obviously, can be both time-consuming and a significant source of errors. In this paper, a modelling method for the management of diagrammatic models with different perspectives on product information is proposed. The aim of this method is to avoid redundant information across models, in order to reduce time needed for ensuring of consistency across models and minimize the chances of errors. The need for information models with different perspectives on the same product information is common in cases that involve design/redesign of complex products and/or the construction of product configurators. Besides a knowledge representation technique, the paper suggests two different solutions for software support, of which the application of one of these in an actual project is described.     

Experimental evaluation of a multivariable adaptive nonlinear predictive controller

A new multivariable adaptive nonlinear predictive controller is designed using a general nonlinear input-output model and variable transformations. The controller is similar in form to typical linear predictive controllers can be tuned analogously or by specifying a single parameters for each controlled variable. In addition, the design procedure is computationally efficient. The new controller is compared to a multi-loop proportional-integral (PI) controller with one-way static decoupling and to an adaptive linear predictive controller through tests on a simulated nonlinear distillation column. The new controller performed well in an experimental application to a multicomponent distillation column.     

不同池化模型的卷积神经网络学习性能研究

目的 基于卷积神经网络的深度学习算法在图像处理领域正引起广泛关注。为了进一步提高卷积神经网络特征提取的准确度,加快参数收敛速度,优化网络学习性能,通过对比不同的池化模型对学习性能的影响提出一种动态自适应的改进池化算法。方法 构建卷积神经网络模型,使用不同的池化模型对网络进行训练,并检验在不同迭代次数下的学习结果。在现有算法准确率不高和收敛速度较慢的情况下,通过使用不同的池化模型对网络进行训练,从而构建一种新的动态自适应池化模型,并研究在不同迭代次数下其对识别准确率和收敛速度的影响。结果 通过对比实验发现,使用动态自适应池化算法的卷积神经网络学习性能最优,在手写数字集上的收敛速度最高可以提升18.55%,而模型对图像的误识率最多可以降低20%。结论 动态自适应池化算法不但使卷积神经网络对特征的提取更加精确,而且很大程度地提高了收敛速度和模型准确率,从而达到优化网络学习性能的目的。这种模型可以进一步拓展到其他与卷积神经网络相关的深度学习算法。     

A hybrid intelligent fuzzy predictive model with simulation for supplier evaluation and selection

Supplier evaluation and selection constitutes a central issue in supply chain management (SCM). However, the data on which to base the corresponding choices in real life problems are often imprecise or vague, which has led to the introduction of fuzzy approaches. Predictive intelligent-based techniques, such as Artificial Neural Network (ANN) and Adaptive Neuro Fuzzy Inference System (ANFIS), have been recently applied in different research fields to model fuzzy multi-criteria decision processes where the understanding and learning of the relationships between the input and output data are the key to select suitable solutions. In this paper, a hybrid ANFIS-ANN model is proposed to assist managers in their supplier evaluation process. After aggregating the data set through the Analytical Hierarchy Process (AHP), the most influential criteria on the suppliers’ performance are determined by ANFIS. Then, Multi-Layer Perceptron (MLP) is used to predict and rank the suppliers’ performance based on the most effective criteria. A case study is presented to illustrate the main steps of the model and show its accuracy in prediction. A battery of parametric tests and sensitivity analyses has been implemented to evaluate the overall performance of several models based on different effective criteria combinations.