An exact confidence set for two binomial proportions and exact unconditional confidence intervals for the difference and ratio of proportions

An exact joint confidence set is proposed for two binomial parameters estimated from independent samples. Its construction relies on inverting the minimum volume test, a two-dimensional analogue of Sterne’s test for a single probability. The algorithm involves computer-intensive exact computation based on binomial probabilities. The proposed confidence set has good coverage properties and it performs much better than the likelihood-based confidence set for the same problem. Applying the principle of intersection-union tests, the method can be used to derive exact tests and confidence intervals for functions of the two binomial parameters. Based on this, new exact unconditional two-sided confidence intervals are proposed for the risk difference and risk ratio. The performance of the new intervals is comparable to that of certain well-known confidence intervals in small samples. Extension of the methods described to two hypergeometric or two Poisson variables is straightforward.     

Generalized F-tests for the multivariate normal mean

Based on Läuter’s [Läuter, J., 1996. Exact t and F tests for analyzing studies with multiple endpoints. Biometrics 52, 964-970] exact t test for biometrical studies related to the multivariate normal mean, we develop a generalized F-test for the multivariate normal mean and extend it to multiple comparison. The proposed generalized F-tests have simple approximate null distributions. A Monte Carlo study and two real examples show that the generalized F-test is at least as good as the optional individual Läuter’s test and can improve its performance in some situations where the projection directions for the Läuter’s test may not be suitably chosen. The generalized F-test could be superior to individual Läuter’s tests and the classical Hotelling T²-test for the general purpose of testing the multivariate normal mean. It is shown by Monte Carlo studies that the extended generalized F-test outperforms the commonly-used classical test for multiple comparison of normal means in the case of high dimension with small sample sizes.     

Assumption adequacy averaging as a concept for developing more robust methods for differential gene expression analysis

The concept of assumption adequacy averaging is introduced as a technique for developing more robust methods that incorporate assessments of assumption adequacy into the analysis. The concept is illustrated by using it to develop a method that averages results from the t-test and nonparametric rank-sum test with weights obtained from using the Shapiro-Wilk test to test the assumption of normality. Through this averaging process, the proposed method is able to rely more heavily on the statistical test that the data suggests is superior for each individual gene. Subsequently, this method developed by assumption adequacy averaging outperforms its two component methods (the t-test and rank-sum test) in a series of traditional and bootstrap-based simulation studies. The proposed method showed greater concordance in gene selection across two studies of gene expression in acute myeloid leukemia than did the t-test or rank-sum test. An R routine for implementing the method is available from www.stjuderesearch.org/depts/biostats.     

A new framework for identifying differentially expressed genes

Microarrays have been widely used to classify cancer samples and discover the biological types, for example tumor versus normal phenotypes in cancer research. One of the challenging scientific tasks in the post-genomic epoch is how to identify a subset of differentially expressed genes from thousands of genes in microarray data which will enable us to understand the underlying molecular mechanisms of diseases, accurately diagnosing diseases and identifying novel therapeutic targets. In this paper, we propose a new framework for identifying differentially expressed genes. In the proposed framework, genes are ranked according to their residuals. The performance of the framework is assessed through applying it to several public microarray data. Experimental results show that the proposed method gives more robust and accurate rank than other statistical test methods, such as t-test, Wilcoxon rank sum test and KS-test. Another novelty of the method is that we design an algorithm for selecting a small subset of genes that show significant variation in expression (“outlier” genes). The number of genes in the small subset can be controlled via an alterable window of confidence level. In addition, the results of the proposed method can be visualized. By observing the residual plot, we can easily find genes that show significant variation in two groups of samples and learn the degrees of differential expression of genes. Through a comparison study, we found several “outlier” genes which had been verified in previous biological experiments while they were either not identified by other methods or had lower ranks in standard statistical tests.     

On the robust detection of edges in time series filtering

Abrupt shifts in the level of a time series represent important information and should be preserved in statistical signal extraction. Various rules for detecting level shifts that are resistant to outliers and which work with only a short time delay are investigated. The properties of robustified versions of the t-test for two independent samples and its non-parametric alternatives are elaborated under different types of noise. Trimmed t-tests, median comparisons, robustified rank and ANOVA tests based on robust scale estimators are compared.     

Multi-aspect permutation tests in shape analysis with small sample size

Inferential methods known in the shape analysis literature make use of configurations of landmarks optimally superimposed using a least-squares procedure or analyze matrices of interlandmark distances. For example, in the two independent sample case, a practical method for comparing the mean shapes in the two groups is to use the Procrustes tangent space coordinates, if data are concentrated, calculate the Mahalanobis distance and then the Hotelling T²-test statistic. Under the assumption of isotropy, another simple approach is to work with statistics based on the squared Procrustes distance and then consider the Goodall F-test statistic. Despite their widespread use, on the one hand it is well known that Hotelling’s T²-test may not be very powerful unless there are a large number of observations available, and on the other hand the underlying model required by Goodall’s F-test is very restrictive. For these reasons, an extension of the nonparametric combination (NPC) methodology to shape analysis is proposed. Focussing on the two independent sample case, through a comparative simulation study and an application to the Mediterranean monk seal skulls dataset, the behaviour of some nonparametric permutation tests has been evaluated, showing that the proposed tests are very powerful, for both balanced and unbalanced sample sizes.     

A permutation test motivated by microarray data analysis

We introduce a nonparametric test intended for large-scale simultaneous inference in situations where the utility of distribution-free tests is limited because of their discrete nature. Such situations are frequently dealt with in microarray analysis where the number of tests is much larger than the sample size. The proposed test statistic is based on a certain distance between the distributions from which the samples under study are drawn. In a simulation study, the proposed permutation test is compared with permutation counterparts of the t-test and the Kolmogorov–Smirnov test. The usefulness of the proposed test is discussed in the context of microarray gene expression data and illustrated with an application to real datasets.     

A framework for modelling overdispersed count data, including the Poisson-shifted generalized inverse Gaussian distribution

A variety of methods of modelling overdispersed count data are compared. The methods are classified into three main categories. The first category are ad hoc methods (i.e. pseudo-likelihood, (extended) quasi-likelihood, double exponential family distributions). The second category are discretized continuous distributions and the third category are observational level random effects models (i.e. mixture models comprising explicit and non-explicit continuous mixture models and finite mixture models). The main focus of the paper is a family of mixed Poisson distributions defined so that its mean μ is an explicit parameter of the distribution. This allows easier interpretation when μ is modelled using explanatory variables and provides a more orthogonal parameterization to ease model fitting. Specific three parameter distributions considered are the Sichel and Delaporte distributions. A new four parameter distribution, the Poisson-shifted generalized inverse Gaussian distribution is introduced, which includes the Sichel and Delaporte distributions as a special and a limiting case respectively. A general formula for the derivative of the likelihood with respect to μ, applicable to the whole family of mixed Poisson distributions considered, is given. Within the framework introduced here all parameters of the distributions are modelled as parametric and/or nonparametric (smooth) functions of explanatory variables. This provides a very flexible way of modelling count data. Maximum (penalized) likelihood estimation is used to fit the (non)parametric models.     

The correction of a two sample sequential t-test developed by Hajnal

The two-sample sequential t-test developed by Hajnal is one of the few existing two-sample sequential tests for a continuous response variable which may be useful for clinical trials. Simulation investigations of this test show that its boundaries do not give exactly the stated significance level and power. Other simulation experiments are the basis for empirical correction of these boundaries.     

A FORTRAN IV program for comparing ranking algorithms in quantitative biostratigraphy

A program for evaluating the performance of competing ranking algorithms in stratigraphic paleontology is presented. The program (1) generates a hypothetical, and thus known, succession of taxa in time and (2) simulates their succession in strata at several local sample sites. If desired, (1) and (2) may be repeated for several (=50 or 100 for example) iterations and the local site data for each sent to two user routines for inferred rankings (inferred succession of events in time). First data for first and last occurrences (fads and lads) taken together, then data for for lads-only, then data for fads-only is sent. For each submission of data to a user routine, Kendall rank correlation coefficients and Spearman coefficients are computed comparing the inferred rankings generated by the user routine with the known succession of events in time. The performance of two competing ranking algorithms may be compared by (1) obtaining for each submitted dataset the differences between corresponding Kendall (and/or Spearman) coefficients computed for the two algorithms, and (2) testing the observed differences for statistical significance. A simple two-sided t-test may be used to test whether the observed mean difference between two corresponding coefficients differs significantly from zero; if ct-tests are performed, the level of significance of each should be set to alpha/c to obtain a maximum experimentwise error rate of less than alpha. The program is used to compare three ranking algorithms provided by Agterberg and Nel (1982a, b) as well as to determine whether the algorithms work as well for datasets combining lads and fads vs datasets for lads-only or fads-only. Agterberg and Nel's Presorting algorithm performed better than their Ranking or Scaling algorithm. All three performed slightly but significantly better on data for lads-only or fads-only as opposed to combined data.     

Score tests for zero-inflated generalized Poisson mixed regression models

Zero-inflated Poisson (ZIP) regression model is a popular approach to the analysis of count data with excess zeros. For correlated count data where the observations are either repeated or clustered outcomes from individual subjects, ZIP mixed regression model may be appropriate. However, ZIP model may often fail to fit such data either because of over-dispersion or because of under-dispersion in relation to the Poisson distribution. In this paper, we extend the ZIP mixed regression model to zero-inflated generalized Poisson (ZIGP) mixed regression model, where the base-line discrete distribution is generalized Poisson (GP) distribution, which is a natural extension of standard Poisson distribution. Furthermore, the random effects are considered in both zero-inflated and GP components throughout the paper. An EM algorithm for estimating parameters is proposed based on the best linear unbiased prediction-type (BLUP) log-likelihood and the residual maximum likelihood (REML). Meanwhile, several score tests are presented for testing the ZIP mixed regression model against the ZIGP mixed regression model, and for testing the significance of regression coefficients in zero-inflation and generalized Poisson portion. A numerical example is given to illustrate our methodology and the properties of score test statistics are investigated through Monte Carlo simulations.     

A flexible approximate likelihood ratio test for detecting differential expression in microarray data

Identifying differentially expressed genes in microarray data has been studied extensively and several methods have been proposed. Most popular methods in the study of gene expression microarray data analysis rely on normal distribution assumption and are based on a Wald statistic. These methods may be inefficient when expression levels follow a skewed distribution. To deal with possible violations of the normality assumption, we propose a method based on Generalized Logistic Distribution of Type II (GLDII). The motivation behind this distributional assumption is to allow longer tails than normal distribution. This is important in analyzing gene expression data since extreme values are common in such experiments. The shape parameter for GLDII allows flexibility in modeling a wide range of distributions. To simplify the computational complexity involved in carrying out Likelihood Ratio (LR) tests for several thousands of genes, an Approximate LR Test (ALRT) is proposed. We also generalize the two-class ALRT method to multi-class microarray data. The performance of the ALRT method under the GLDII assumption is compared to methods based on Wald-type statistics using simulation. The results from the simulations show that our method performs quite well compared to the significance analysis of microarrays (SAM) approach using standardized Wilcoxon rank statistics and the empirical Bayes (E-B) t-statistics. Our method is also less sensitive to extreme values. We illustrate our method using two publicly available gene expression data sets.     

Rule extraction by fuzzy modeling algorithm for fuzzy logic control of cisatracurium as a neuromuscular block

This paper provides two rule bases to control administration of cisatracurium, a non-depolarizing neuromuscular blocking agent. One rule base is extracted from the objective approach of fuzzy modeling algorithm (FMA), and the other is from the subjective approach of experts’ clinical experience. First, we established the data-acquisition system to record the manual neuromuscular block control during surgery. After collecting 15 patients data control by cisatracurium, we extracted six rules from these data via FMA. Another rule base also had six rules from experts with clinical anesthesia experience. Each rule-base was combined with three rules regarding the safety of the fuzzy controller. To compare their performance through simulations, we used the patient model established by our previous study which is a combination model consisting of a three-compartment mathematical model based on pharmacokinetics, and the Hill equation based on pharmacodynamics. In order to test the differences between these two rule-bases, the simulation used four disturbances: the different set points, the control interval strategy, the tolerance of noise effect, and the tolerance of delay time effect. The simulation shows that the FMA could successfully extract the fuzzy rules from the clinical data, and its control error is smaller than expert rules for different set point tests. However, the control error is increased and becomes worse when the set points are raised, which means that these two rule-bases are not appropriate to control the higher set points (i.e. T1% of 40 or higher). The t-test also shows that these two rule-bases performance of different set points have significant differences (p<0.05). Moreover, the results for control interval tests show that strategy has a significant influence, especially in reducing the standard deviation of control error. However, in simulations, these two rule-bases are not affected by noise disturbance, and the delay time affects only the overshoot for these two rule-bases.     

A score test for zero-inflation in multilevel count data

The zero-inflated Poisson regression (ZIP) in many situations is appropriate for analyzing multilevel correlated count data with excess zeros. In this paper, a score test for assessing ZIP regression against Poisson regression in multilevel count data with excess zeros is developed. The sampling distribution and power of the score statistic test is evaluated using a simulation study. The results show that under a wide range of conditions, the score statistic performs satisfactorily. Finally, the use of the score test is illustrated on DMFT index data of children 7-8 years old.     

Type I multivariate zero-inflated Poisson distribution with applications

Motivated from the stochastic representation of the univariate zero-inflated Poisson (ZIP) random variable, the authors propose a multivariate ZIP distribution, called as Type I multivariate ZIP distribution, to model correlated multivariate count data with extra zeros. The distributional theory and associated properties are developed. Maximum likelihood estimates for parameters of interest are obtained by Fisher’s scoring algorithm and the expectation–maximization (EM) algorithm, respectively. Asymptotic and bootstrap confidence intervals of parameters are provided. Likelihood ratio test and score test are derived and are compared via simulation studies. Bayesian methods are also presented if prior information on parameters is available. Two real data sets are used to illustrate the proposed methods. Under both AIC and BIC, our analysis of the two data sets supports the Type I multivariate zero-inflated Poisson model as a much less complex alternative with feasibility to the existing multivariate ZIP models proposed by Li et al. (Technometrics, 29–38, Vol 41, 1999).     

Feature selection in bankruptcy prediction

For many corporations, assessing the credit of investment targets and the possibility of bankruptcy is a vital issue before investment. Data mining and machine learning techniques have been applied to solve the bankruptcy prediction and credit scoring problems. As feature selection is an important step to select more representative data from a given dataset in data mining to improve the final prediction performance, it is unknown that which feature selection method is better. Therefore, this paper aims at comparing five well-known feature selection methods used in bankruptcy prediction, which are t-test, correlation matrix, stepwise regression, principle component analysis (PCA) and factor analysis (FA) to examine their prediction performance. Multi-layer perceptron (MLP) neural networks are used as the prediction model. Five related datasets are used in order to provide a reliable conclusion. Regarding the experimental results, the t-test feature selection method outperforms the other ones by the two performance measurements.     

Performance of feature-selection methods in the classification of high-dimension data

Contemporary biological technologies produce extremely high-dimensional data sets from which to design classifiers, with 20,000 or more potential features being common place. In addition, sample sizes tend to be small. In such settings, feature selection is an inevitable part of classifier design. Heretofore, there have been a number of comparative studies for feature selection, but they have either considered settings with much smaller dimensionality than those occurring in current bioinformatics applications or constrained their study to a few real data sets. This study compares some basic feature-selection methods in settings involving thousands of features, using both model-based synthetic data and real data. It defines distribution models involving different numbers of markers (useful features) versus non-markers (useless features) and different kinds of relations among the features. Under this framework, it evaluates the performances of feature-selection algorithms for different distribution models and classifiers. Both classification error and the number of discovered markers are computed. Although the results clearly show that none of the considered feature-selection methods performs best across all scenarios, there are some general trends relative to sample size and relations among the features. For instance, the classifier-independent univariate filter methods have similar trends. Filter methods such as the t-test have better or similar performance with wrapper methods for harder problems. This improved performance is usually accompanied with significant peaking. Wrapper methods have better performance when the sample size is sufficiently large. ReliefF, the classifier-independent multivariate filter method, has worse performance than univariate filter methods in most cases; however, ReliefF-based wrapper methods show performance similar to their t-test-based counterparts.     

Joint maximum likelihood estimation of unit root testing equations and GARCH processes: Some finite-sample issues

In recent research [B. Seo, Distribution theory for unit root tests with conditional heteroskedasticity, J. Econometrics 91 (1999) 113–144] has suggested that the examination of the unit root hypothesis in series exhibiting GARCH behaviour should proceed via joint maximum likelihood (ML) estimation of the unit root testing equation and GARCH process. The results presented show the asymptotic distribution of the resulting ML t-test to be a mixture of the Dickey–Fuller and standard normal distributions. In this paper, the relevance of these asymptotic arguments is considered for the finite samples encountered in empirical research. In particular, the influences of sample size, alternative values of the parameters of the GARCH process and the use of the Bollerslev–Wooldridge covariance matrix estimator upon the finite-sample distribution of the ML t-statistic are explored. It is shown that the resulting critical values for the ML t-statistic are similar to those of the Dickey–Fuller distribution rather than the standard normal, unless a large sample size and empirically unrealistic values of the volatility parameter of the GARCH process are considered. Use of the Bollerslev–Wooldridge standard covariance matrix estimator exaggerates this finding, causing a leftward shift in the finite-sample distribution of the ML t-statistic. The results of the simulation analysis are illustrated via an application to U.S. short term interest rates.     

Optimizing area under the ROC curve using semi-supervised learning

Receiver operating characteristic (ROC) analysis is a standard methodology to evaluate the performance of a binary classification system. The area under the ROC curve (AUC) is a performance metric that summarizes how well a classifier separates two classes. Traditional AUC optimization techniques are supervised learning methods that utilize only labeled data (i.e., the true class is known for all data) to train the classifiers. In this work, inspired by semi-supervised and transductive learning, we propose two new AUC optimization algorithms hereby referred to as semi-supervised learning receiver operating characteristic (SSLROC) algorithms, which utilize unlabeled test samples in classifier training to maximize AUC. Unlabeled samples are incorporated into the AUC optimization process, and their ranking relationships to labeled positive and negative training samples are considered as optimization constraints. The introduced test samples will cause the learned decision boundary in a multi-dimensional feature space to adapt not only to the distribution of labeled training data, but also to the distribution of unlabeled test data. We formulate the semi-supervised AUC optimization problem as a semi-definite programming problem based on the margin maximization theory. The proposed methods SSLROC1 (1-norm) and SSLROC2 (2-norm) were evaluated using 34 (determined by power analysis) randomly selected datasets from the University of California, Irvine machine learning repository. Wilcoxon signed rank tests showed that the proposed methods achieved significant improvement compared with state-of-the-art methods. The proposed methods were also applied to a CT colonography dataset for colonic polyp classification and showed promising results.¹