Multivariate Analyses of a Production Formulation Optimization Experiment

Abstract

Three prominent multivariate statistical analyses, canonical correlation analysis (CCA), principal component analysis (PCA) and CAS-Regression analysis (CAS-R) are appropriately applied to the formulation optimization data associated with Product-T for determining a set of key excipient/process variables and a set of key response variables to be used in monitoring the future performance of the optimizated formula. CCA which considers both sets of variables simultaneously in a single analysis, successfully delineated two key parameters, one for each set. PCA which considers only the response variables concurred with the CCA results and CAS-R which considers each response variable separately also concurred. Even though CCA is a predominant technique, adjunct results of PCA and CAS-R could be supplemented for a comprehensive interpretation. It is recommended that all three analyses be carried out and interpreted appropriately.     

Order Statistics from the Gamma Distribution

This paper deals with order statistics from a gamma or χ² (Pearson Type III) distribution. Expressions are derived for the moments of an order statistic and for the covariance between two order statistics. A table of moments (about the origin as well as the mean) is presented. Equations are derived which are used to solve for the percentage points of the order statistics and a table of the percentage points is given. The problem of the best linear unbiased estimate of the scale parameter of the distribution which is based on order statistics is, briefly, discussed for the censored (type II) case. The modal value of an order statistic is derived and a table of these values is given for the cases of the smallest and the largest order statistics. Applications ‘to life-tests, extreme values, reliability and maintenance are described and illustrated in some cases.     

A general near-exact distribution theory for the most common likelihood ratio test statistics used in Multivariate Analysis

In this paper we first show how the exact distributions of the most common likelihood ratio test (l.r.t.) statistics, that is, the ones used to test the independence of several sets of variables, the equality of several variance-covariance matrices, sphericity, and the equality of several mean vectors, may be expressed as the distribution of the product of independent Beta random variables or the product of a given number of independent random variables whose logarithm has a Gamma distribution times a given number of independent Beta random variables. Then, we show how near-exact distributions for the logarithms of these statistics may be expressed as Generalized Near-Integer Gamma distributions or mixtures of these distributions, whose rate parameters associated with the integer shape parameters, for samples of size n, all have the form (n−j)/n for j=2,…,p, where for three of the statistics, p is the number of variables involved, while for the other one, it is the sum of the number of variables involved and the number of mean vectors being tested. What is interesting is that the similarities exhibited by these statistics are even more striking in terms of near-exact distributions than in terms of exact distributions. Moreover all the l.r.t. statistics that may be built as products of these basic statistics also inherit a similar structure for their near-exact distributions. To illustrate this fact, an application is made to the l.r.t. statistic to test the equality of several multivariate Normal distributions.     

Analysis of Bivariate Ordered Categorical Data by Normal Models

An extension of a bivariate probit model is presented for bivariate polychotomous ordered categorical responses. A linear model is considered on the assumption that the observed categorical variables are manifestations of the latent continuous variables having bivariate normal distributions. Method of inference and analytical procedures are given on the basis of the maximum likelihood procedure. Various applications are discussed and illustrated with examples. The model can be applied to analysis of association, paired comparison, matched pair experiments, testing homogeneity of marginal distributions and symmetry of a square table, factorial analysis with bivariate ordered categorical responses, and so on.

     

A Goodness-of-Fit Test for the Two Parameter vs. Three Parameter Weibull; Confidence Bounds for ThreshoId

An investigation is described herein of modifications of the two-parameter Weibullgoodness-of-fit test of Mann, Scheuer and Fertig (1973). It is assumed that the sole alternative of interest is any three-parameter Weibull distribution. The power of candidate test statistics is investigated, therefore, only under various three-parameter Weibull alternative hypotheses.

It is found that for a fixed selection of gaps (differences of adjacent order statistics) used in the numerator and in the denominator of the approximately F dist ributed test statistic, nothing is gained by weighting the gaps in order to minimize the variances and thus to maximize the numbers of degrees of freedom.

A test statistic which is a modified version of that of Mann, Scheuer and Fertig is shown to have higher power under three-parameter Weibull alternatives, and a simple method for approximating critical values of the test statistic is described. The test statistic is shown to be a monotone function of an unknown threshold (location) parameter for the three-parameter Weibull model. Hence, the methodology described for testing for a zero threshold parameter can be used to obtiain a confidence interval for this parameter. Methods for combining life-test data for application to progressively censored samples are also described.     

Mixed Variable Optimization of the Number and Composition of Heat Intercepts in a Thermal Insulation System

In the literature, thermal insulation systems with a fixed number of heat intercepts have been optimized with respect to intercept locations and temperatures. The number of intercepts and the types of insulators that surround them were chosen by parametric studies. This was because the optimization methods used could not treat such categorical variables. Discrete optimization variables are categorical if the objective function or the constraints can not be evaluated unless the variables take one of a prescribed enumerable set of values. The key issue is that categorical variables can not be treated as ordinary discrete variables are treated by relaxing them to continuous variables with a side constraint that they be discrete at the solution.A new mixed variable programming (MVP) algorithm makes it possible to optimize directly with respect to mixtures of discrete, continuous, and categorical decision variables. The result of applying MVP is shown here to give a 65% reduction in the objective function over the previously published result for a thermal insulation model from the engineering literature. This reduction is largely because MVP optimizes simultaneously with respect to the number of heat intercepts and the choices from a list of insulator types as well as intercept locations and temperatures. The main purpose of this paper is to show that the mixed variable optimization algorithm can be applied effectively to a broad class of optimization problems in engineering that could not be easily solved with earlier methods.     

Performances of Randomization Test for Comparing Longitudinal Ordered Categorical Outcome

In this paper, we consider methods to analyse longitudinal ordered categorical data. Several methods for analysing such data have been proposed, but they are often limited in their ranges of application. We propose to treat the ordered categorical data as if they are continuous ones by scoring ordered categories, and to apply randomization test to them. In addition, we recommend joint use of time continuous type method (Zerbe, 1979) and time-discrete type method (Raz, 1989) when both treatment main effect and treatment-by-time interaction are to be tested by the randomization test. By a simulation study, we evaluated performances of our recommending approach for the tests of treatment main effect and treatment-by-time interaction based on longitudinal ordered categorical data. We defined effects of these two factors on an underlying continuous scale in the simulation study. As a result, it was found that the performances of these two tests are satisfactory with respect to type I error rate and power. However, in the case where extremely large (or small) score was given to the maximum (or the minimum) category, the powers of these two tests were slightly low.

     

An Error Rate Comparison of Classification Methods with Continuous Explanatory Variables

In the fields of statistics and computer science a wide variety of methodologies exist for solving the traditional classification problem. This study will compare the error rates for various methods under a variety of conditions when the explanatory variables are continuous. The methods under considerations are neural networks, classical discriminant analysis, and two different approaches to decision trees. Training and testing sets are utilized to estimate the error rates of these methods for different numbers of sample sizes, number of explanatory variables, and the number of classes in the dependent variable. These error rates will be used to draw generalized conclusions about the relative efficiencies of the techniques.     

Optimal significance analysis of microarray data in a class of tests whose null statistic can be constructed

Microarray data often consist of a large number of genes and a small number of replicates. We have examined testing the null hypothesis of equality of mean for detecting differentially expressed genes. The p-value for each gene is often estimated using permutation samples not only for the target gene but also for other genes. This method has been widely used and discussed. However, direct use of the permutation method for the p-value estimation may not work well, because two types of genes are mixed in the sample; some genes are differentially expressed, whereas others are not. To overcome this difficulty, various methods for appropriately generating null permutation samples have been proposed. In this paper, we consider two classes of test statistics that are naturally modified to null statistics. We then obtain the uniformly most powerful (UMP) unbiased tests among these classes. If the underlying distribution is symmetric, the UMP unbiased test statistic is similar to that proposed by Pan (Bioinformatics 19:1333?C1340, 2003). Under another condition, the UMP unbiased test statistic has a different formula with one more degree of freedom and therefore is expected to give a more powerful test and a more accurate p-value estimation from a modified null statistic. In microarray data, because the number of replicates is often small, differences in the degree of freedom will produce large effects on the power of test and the variance of the p-value estimation. Some simulation studies and real data analyses are illustrated to investigate the performances of the methods.     

An adaptive step-down procedure for fault variable identification

In a process with a large number of process variables (high-dimensional process), identifying which variables cause an out-of-control signal is a challenging issue for quality engineers. In this paper, we propose an adaptive step-down procedure using conditional T² statistic for fault variable identification. While existing procedures focus on selecting variables that have strong evidence of a change, the proposed step-down procedure selects a variable having the weakest evidence of a change at each step based on the variables that are selected in previous steps. The information of selected unchanged variables is effectively utilised in obtaining a powerful conditional T² test statistic for identifying the changed elements of the mean vector. The proposed procedure is designed to utilise the correlation information between fault and non-fault variables for the efficient fault variables identification. Further, the simulation results show that the proposed procedure has the better diagnostic performance compared with existing methods in terms of fault variable identification and computational complexity, especially when the number of the variables is high and the number of fault variables is small.     

SEQUENTIAL WORK SAMPLING TESTS

In this paper a test method for time study developed from a theoretical basis of sequential analysis of mathematical statistics is presented. This method admits a statistical decision on the regions which contain a certain time ratio.

In the course of the test process, the instantaneous observations of the activities made at the work places are assigned by small integers which yield a simple sequential statistic. The observations need not be recorded. The sample size is, on the average, considerably smaller than that of comparable studies by normal work sampling.

The design and application of sequential work sampling tests are described with regard to the estimation method of work sampling and their most important features are discussed.     

Evaluation of the 0.3 logits screening criterion in common item equating

A number of state assessment programs that employ Rasch-based common item equating procedures estimate the equating constant with only those common items for which the two tests' Rasch item difficulty parameter estimates differ by less than 0.3 logits. The results of this study presents evidence that this practice results in an inflated probability of incorrectly dropping an item from the common item set if the number of examinees is small (e.g., 500 or less) and the reverse if the number of examinees is large (e.g., 5000 or more). An asymptotic experiment-wise error rate criterion was algebraically derived. This same criterion can also be applied to the Mantel-Haenszel statistic. Bonferroni test statistics were found to provide excellent approximations to the (asymptotically) exact test statistics.     

Correlation Analysis for Simultaneously Controlling Reliability and Validity

The purpose of the present study is to propose a procedure for correlation analysis of several (especially two) sets of variables, which includes canonical correlation analysis, principal component analysis, and multiple regression analysis as a special case. The proposed method derives components from each set of variables which maximize the weighted geometric mean of two types of indicators: one is the contribution rate of the components for their original variables, the other is the squared correlation between the components. In terms of the test theory, the former are indicators of reliability and the latter are indicators of concurrent validity. Through the numerical examples applying this method to the data of two Japanese language personality inventory, the method is shown to be particularly useful when determining the weights for test items.

     

Regime Testimation in a Time Sequence

Two years of irregularly spaced measurements of seawater temperature at 200 feet at a fixed North Pacific Ocean location are considered. The data are heteroscedastic in time and the average path jumps about erratically. We seek points of significant change in the data-generating process and wish to study their nature.

The set of all properties characterizing a data-generating process is its regime. Simultaneous testing and estimation is testimation. We must testimate abrupt changes in the regime as a whole instead of property by property. A moving mean square of error values is posed as a statistic and its relation to x ² and F derived. Methods of testimation and decomposition of the statistic into proportions due to each possible cause are obtained. The pattern of behavior of the statistic which identifies an outlier is considered. It is possible for two properties in the regime to change jointly such that one change obscures the other. Methods to detect and solve this problem are derived.

The temperature data are analyzed by regime testimation methods.     

Clustering Nominal data with Equivalent Categories

The problem considered in the present paper is how to cluster data of nominal measurement level, where the categories of the variables are equivalent (the variables are replications of each other). One suitable technique to obtain such a clustering is latent class analysis (LCA) with equality restrictions on the conditional probabilities. As an alternative, a less well known technique is introduced: GROUPALS. This is an algorithm for the simultaneous scaling (by multiple correspondence analysis) and clustering of categorical variables. Equality restrictions on the category quantifications were incorporated in the algorithm, to account for equivalent categories. In two simulation studies, the clustering performance was assessed by measuring the recovery of true cluster membership of the individuals. The effect of several systematically varied data features was studied. Restricted LCA obtained good to excellent cluster recovery results. Restricted GROUPALS approximated this optimal performance reasonably well, except when underlying classes were very different in size.

     

An EWMA signed ranks control chart with reliable run length performances

During the design phase of a control chart, the determination of its exact run length properties plays a vital role for its optimal operation. Markov chain or integral equation methods have been extensively applied in the design phase of conventional control charts. However, for distribution-free schemes, due to the discrete nature of the statistics being used (such as the sign or the Wilcoxon signed rank statistics, for instance), it is impossible to accurately compute their run length properties. In this work, a modified distribution-free phase II exponentially weighted moving average (EWMA)-type chart based on the Wilcoxon signed rank statistic is considered and its exact run length properties are discussed. A continuous transformation of the Wilcoxon signed rank statistic, combined with the classical Markov chain method, is used for the determination of the average run length in the in- and out-of control cases. Moreover, its exact performance is derived without any knowledge of the distribution of sample observations. Finally, an illustrative example is provided showing the practical implementation of our proposed chart.     

Life-Test Sampling Plans for Two-Parameter Weibull Populations

In this paper, we give variables sampling plans for items whose failure times are distributed as either extreme-value variates or Weibull variates (the logarithms of which are from an extreme-value distribution). Tables applying to acceptance regions and operating characteristics for sample size n, ranging from 3 to 18, are given. The tables allow for Type II censoring, with censoring number r ranging from 3 to n. In order to fix the maximum time on test, the sampling plan also allows for Type I censoring.

Acceptance/rejection is based upon a statistic incorporating best linear invariant estimates, or, alternatively, maximum likelihood estimates of the location and scale parameters of the underlying extreme value distribution. The operating characteristics are computed using an approximation discussed by Fertig and Mann (1980).     

A New Measure of Misfit for Covariance Structure Models

Various fit indices exist in structural equation models. Most of these indices are related to the noncentrality parameter (NCP) of the chi-square distribution that the involved test statistic is implicitly assumed to follow. Existing literature suggests that few statistics can be well approximated by chi-square distributions. The meaning of the NCP is not clear when the behavior of the statistic cannot be described by a chi-square distribution. In this paper we define a new measure of model misfit (MMM) as the difference between the expected values of a statistic under the alternative and null hypotheses. This definition does not need to assume that the population covariance matrix is in the vicinity of the proposed model, nor does it need for the test statistic to follow any distribution of a known form. The MMM does not necessarily equal the discrepancy between the model and the population covariance matrix as has been assumed in existing literature. Bootstrap approaches to estimating the MMM and a related quantity are developed. An algorithm for obtaining bootstrap confidence intervals of the MMM is constructed. Examples with practical data sets contrast several measures of model misfit. The quantile-quantile plot is used to illustrate the unrealistic nature of chi-square distribution assumptions under either the null or an alternative hypothesis in practice.

     

A NEW APPROACH TO NONLINEAR MIXED DISCRETE PROGRAMMING PROBLEMS

Abstract

In this paper a novel method is developed for the numerical solution of a non-linear mixed discrete programming problem. The problem is first transformed into another mixed zero-one discrete optimization problem by replacing each of the discrete variables by a set of new variables with a linear constraint. Each of the new discrete variable sets is then made continuous on [0,1] by introducing a quadratic constraints. Thus the original mixed discrete problem is transformed into a continuous nonlinear optimization problem with constraints. The numerical experiments, performed to demonstrate the effectiveness of the method, show that the method is superior to the previous ones.     

CUSUM Schemes for Monitoring Prespecified Changes in Linear Profiles

Because of the characteristics of a system or process, several prespecified changes may happen in some statistical process control applications. Thus, one possible and challenging problem in profile monitoring is detecting changes away from the ‘normal’ profile toward one of several prespecified ‘bad’ profiles. In this article, to monitor the prespecified changes in linear profiles, two two‐sided cumulative sum (CUSUM) schemes are proposed based on Student's t‐statistic, which use two separate statistics and a single statistic, respectively. Simulation results show that the CUSUM scheme with a single statistic uniformly outperforms that with two separate statistics. Besides, both CUSUM schemes perform better than alternative methods in detecting small shifts in prespecified changes, and become comparable on detecting moderate or large shifts when the number of observations in each profile is large. To overcome the weakness in the proposed CUSUM methods, two modified CUSUM schemes are developed using z‐statistic and studied when the in‐control parameters are estimated. Simulation results indicate that the modified CUSUM chart with a single charting statistic slightly outperforms that with two separate statistics in terms of the average run length and its standard deviation. Finally, illustrative examples indicate that the CUSUM schemes are effective. Copyright © 2016 John Wiley & Sons, Ltd.