Bootstrapping divergence statistics for testing homogeneity in multinomial populations

We consider the problem of testing the equality of ν$\nu$ (ν≥2$\nu \ge 2$) multinomial populations, taking as test statistic a sample version of an f-dissimilarity between the populations, obtained by the replacement of the unknown parameters in the expression of the f-dissimilarity among the theoretical populations, by their maximum likelihood estimators. The null distribution of this test statistic is usually approximated by its limit, the asymptotic null distribution. Here we study another way to approximate it, the bootstrap. We show that the bootstrap yields a consistent distribution estimator. We also study by simulation the finite sample performance of the bootstrap distribution and compare it with the asymptotic approximation. From the simulations it can be concluded that it is worth calculating the bootstrap estimator, because it is more accurate than the approximation yielded by the asymptotic null distribution which, in addition, cannot always be exactly computed.     

Testing equality of covariance matrices when data are incomplete

In the statistics literature, a number of procedures have been proposed for testing equality of several groups’ covariance matrices when data are complete, but this problem has not been considered for incomplete data in a general setting. This paper proposes statistical tests for equality of covariance matrices when data are missing. A Wald test (denoted by T₁), a likelihood ratio test (LRT) (denoted by R), based on the assumption of normal populations are developed. It is well-known that for the complete data case the classic LRT and the Wald test constructed under the normality assumption perform poorly in instances when data are not from multivariate normal distributions. As expected, this is also the case for the incomplete data case and therefore has led us to construct a robust Wald test (denoted by T₂) that performs well for both normal and non-normal data. A re-scaled LRT (denoted by R^*) is also proposed. A simulation study is carried out to assess the performance of T₁, T₂, R, and R^* in terms of closeness of their observed significance level to the nominal significance level as well as the power of these tests. It is found that T₂ performs very well for both normal and non-normal data in both small and large samples. In addition to its usual applications, we have discussed the application of the proposed tests in testing whether a set of data are missing completely at random (MCAR).     

CHITEST: A Monte-Carlo computer program for contingency table tests

CHITEST is an interactive FORTRAN 77 program that uses Monte-Carlo methods to test the null hypothesis that the row and column factors of a r-by-k contingency table are independent of each other. The program optionally performs the test for two populations: (1) the population of tables that have the same row and column marginal frequencies as the observed table; and (2) the population of tables that have the same total frequency as the observed table. The test does not require the expected cell frequencies to be large values—a requirement necessary for the standard chi-square test of independence to be valid. The program also will test the goodness-of-fit of an empirical distribution to a discrete theoretical distribution, and this test does not require large expected values for the theoretical distribution.     

Near concurrent MIR, SSM/T-2, and SSM/I observations over snow-covered surfaces

The airborne Millimeter-wave Imaging Radiometer (MIR) and MODIS Airborne Simulator (MAS) measurements over the Arctic region and the Midwest region of the US are used to derive surface emissivities ξ(ν) for three frequencies, ν=89, 150, and 220 GHz, as well as Normalized Difference Snow Index (NDSI) and R87 (0.87-μm reflectance). These derived parameters are compared with parameters estimated from near concurrent measurements made by the SSMI and SSM/T-2 over snow-covered areas. It is shown that the MIR-estimated ξ(ν) values at ν=89 and 150 GHz agree well with those estimated from the SSM/T-2 at ν=91 and 150 GHz, respectively. Low MIR-estimated ξ(ν) values are generally associated with high NDSI and R87 over the snow-covered areas. Over forested areas, more fluctuations in the values of MIR-estimated ξ(ν), NDSI and R87, as well as a reduction in polarization index (PI) at 37 and 85 GHz are observed.Both observations and results from radiative transfer calculations show a change in the difference between brightness temperatures (T_b) at 19 and 37 GHz, as well as PI at 37 and 85 GHz, when measured at satellite altitudes and at the surface. The amplitude of the T_b difference and PI is reduced by about 10-15% from surface to high altitudes when integrated water vapor is ≤1.5 g/cm². This effect is readily correctable and requires consideration when validating satellite retrieval algorithms based on surface and low-elevation aircraft measurements.     

A theory of learning from different domains

Discriminative learning methods for classification perform well when training and test data are drawn from the same distribution. Often, however, we have plentiful labeled training data from a source domain but wish to learn a classifier which performs well on a target domain with a different distribution and little or no labeled training data. In this work we investigate two questions. First, under what conditions can a classifier trained from source data be expected to perform well on target data? Second, given a small amount of labeled target data, how should we combine it during training with the large amount of labeled source data to achieve the lowest target error at test time? We address the first question by bounding a classifier’s target error in terms of its source error and the divergence between the two domains. We give a classifier-induced divergence measure that can be estimated from finite, unlabeled samples from the domains. Under the assumption that there exists some hypothesis that performs well in both domains, we show that this quantity together with the empirical source error characterize the target error of a source-trained classifier. We answer the second question by bounding the target error of a model which minimizes a convex combination of the empirical source and target errors. Previous theoretical work has considered minimizing just the source error, just the target error, or weighting instances from the two domains equally. We show how to choose the optimal combination of source and target error as a function of the divergence, the sample sizes of both domains, and the complexity of the hypothesis class. The resulting bound generalizes the previously studied cases and is always at least as tight as a bound which considers minimizing only the target error or an equal weighting of source and target errors.     

Efficient algorithms to compute Hankel transforms using wavelets

The aim of the paper is to propose two efficient algorithms for the numerical evaluation of Hankel transform of order ν, ν>−1 using Legendre and rationalized Haar (RH) wavelets. The philosophy behind the algorithms is to replace the part xf(x) of the integrand by its wavelet decomposition obtained by using Legendre wavelets for the first algorithm and RH wavelets for the second one, thus representing Fν(y) as a Fourier-Bessel series with coefficients depending strongly on the input function xf(x) in both the cases. Numerical evaluations of test functions with known analytical Hankel transforms illustrate the proposed algorithms.     

An improved ν-twin support vector machine

Twin support vector machine (TSVM) is regarded as a milestone in the development of the powerful SVM. It finds two nonparallel planes by resolving a pair of smaller-sized quadratic programming problems rather than a single large one, which makes the learning speed of TSVM approximately four times faster than that of the standard SVM. However, the empirical risk minimization principle is implemented in the TSVM, so it easily leads to the over-fitting problem and reduces the prediction accuracy of the classifier. ν-TSVM, as a variant of TSVM, also implements the empirical risk minimization principle. To enhance the generalization ability of the classifier, we propose an improved ν-TSVM by introducing a regularization term into the objective function, so there are two parts in the objective function, one of which is to maximize the margin between the two parallel hyper-planes, and the other one is to minimize the training errors of two classes of samples. Therefore the structural risk minimization principle is implemented in our improved ν-TSVM. Numerical experiments on one artificial dataset and nine benchmark datasets show that our improved ν-TSVM yields better generalization performance than SVM, ν-SVM, and ν-TSVM. Moreover, numerical experiments with different proportions of outliers demonstrate that our improved ν-TSVM is robust and stable. Finally, we apply our improved ν-TSVM to two BCI competition datasets, and also obtain better prediction accuracy.     

Maximum likelihood ratio test for the stability of sequence of Gaussian random processes

Based on the Karhunen-Loeve expansion, the maximum likelihood ratio test for the stability of sequence of Gaussian random processes is investigated. The likelihood function is based on the first p scores of eigenfunctions in the Karhunen-Loeve expansion for Gaussian random processes. Though the scores are unobservable, we show that the effect of the difference between scores and their estimators is negligible as the sample size tends to infinity. The asymptotic distribution is proved to be the Gumbel extreme value distribution. Under the alternative the test is shown to be consistent. For different choices of p, simulation results show that the test behaves quite well in finite samples. The test procedure is also applied to the annual temperature data of central England. The results show that the temperatures have risen in the last twenty years, however there is no evidence to show that the autocovariance functions of the temperatures have changed among the range of the observations.     

Generalized likelihood ratio test for varying-coefficient models with different smoothing variables

Varying-coefficient models are popular multivariate nonparametric fitting techniques. When all coefficient functions in a varying-coefficient model share the same smoothing variable, inference tools available include the F-test, the sieve empirical likelihood ratio test and the generalized likelihood ratio (GLR) test. However, when the coefficient functions have different smoothing variables, these tools cannot be used directly to make inferences on the model because of the differences in the process of estimating the functions. In this paper, the GLR test is extended to models of the latter case by the efficient estimators of these coefficient functions. Under the null hypothesis the new proposed GLR test follows the χ²-distribution asymptotically with scale constant and degree of freedom independent of the nuisance parameters, known as Wilks phenomenon. Further, we have derived its asymptotic power which is shown to achieve the optimal rate of convergence for nonparametric hypothesis testing. A simulation study is conducted to evaluate the test procedure empirically.     

Benchmark test of drift-kinetic and gyrokinetic codes through neoclassical transport simulations

Two simulation codes that solve the drift-kinetic or gyrokinetic equation in toroidal plasmas are benchmarked by comparing the simulation results of neoclassical transport. The two codes are the drift-kinetic δf Monte Carlo code (FORTEC-3D) and the gyrokinetic full-f Vlasov code (GT5D), both of which solve radially-global, five-dimensional kinetic equation with including the linear Fokker-Planck collision operator. In a tokamak configuration, neoclassical radial heat flux and the parallel flow relation, which relates the parallel mean flow with radial electric field and temperature gradient, are compared between these two codes, and their results are also compared with the local neoclassical transport theory. It is found that the simulation results of the two codes coincide very well in a wide rage of plasma collisionality parameter ν_∗=0.01∼10 and also agree with the theoretical estimations. The time evolution of radial electric field and particle flux, and the radial profile of the geodesic acoustic mode frequency also coincide very well. These facts guarantee the capability of GT5D to simulate plasma turbulence transport with including proper neoclassical effects of collisional diffusion and equilibrium radial electric field.     

An algorithm for the determination of the vibrational energy distribution in “hot” molecules. Description and test

An algorithm for the determination of the vibrational energy of molecules energized above their threshold for unimolecular reaction is presented. Using easily obtainable experimental data the method performs their deconvolution applying well-established unimolecular reaction concepts. Their mathematical foundations are described as well as its implementation in IBM AT compatible TB BASIC. In the case example the vibrational energy distribution is known (can be calculated thermochemically), and therefore can be used to test the method performance. It is shown that using precise unimolecular reaction parameters and well-defined average energies transferred by collision 〈ΔE〉 the agreement between deconvoluted and thermochemical results is good (i.e. within the uncertainties involved in the thermochemical test calculation).     

Bounded Contractions of Full Trees

Let G be a simple finite connected undirected graph. A contraction φ of G is a mapping from G = G(V, E) toG′ = G′(V′, E′), where G′ is also a simple connected undirected graph, such that if u, ν ∈ V are connected by an edge (adjacent) in G then either φ(u) = φ(ν), or φ(u) and φ(ν) are adjacent in G′. In this paper we are interested in a family of contractions, called bounded contractions, in which ∀ν′ ∈ V′, the degree of ν′ in G′, Deg_G′(ν′), satisfies Deg_G′(ν′) ≤ |φ⁻¹(ν′)|, where φ⁻¹(ν′) denotes the set of vertices in G mapped to ν′ under φ. These types of contractions are useful in the assignment (mapping) of parallel programs to a network of interconnected processors, where the number of communication channels of each processor is small. The main results of this paper are that there exists a partitioning of full m-ary trees that yields a bounded contraction of degree m + 1, i.e., a mapping for which ∀ν′ ∈ V′, |φ⁻¹(ν′)| ≤ m + 1, and that this degree is a lower bound, i.e., there is no mapping of a full m-ary tree such that ∀ν′ ∈ V′, |φ⁻¹(ν′)| ≤ m     

Ein Abbrechkriterium für einen numerischen Algorithmus

A termination criterion by Nickel (Theorem 6, [1]) for guaranteeing the numerical convergence for locally stable and consistent algorithms is generalized. The assumption |x _ν+2,x _ν+1|≤L|x _ν+1,x _ν| (0≤L<1,L real constant) of the approximation sequence {x _ν} to the solution is replaced by the convergence of the progression \(\sum\limits_{v = 1}^\infty {|x,x_{v + 1} |/Q^v (0< Q< 1)} \) . Therefore the theorem of this paper is applicable to a large number of numerical procedures, for which untill now no termination criterion has been known (for example: Rombergprocedure). In particular this weakening is important for the computation of approximation solutions for integral equations.     

Stochastic volatility models with leverage and heavy-tailed distributions: A Bayesian approach using scale mixtures

This paper studies a heavy-tailed stochastic volatility (SV) model with leverage effect, where a bivariate Student-t distribution is used to model the error innovations of the return and volatility equations. Choy et al. (2008) studied this model by expressing the bivariate Student-t distribution as a scale mixture of bivariate normal distributions. We propose an alternative formulation by first deriving a conditional Student-t distribution for the return and a marginal Student-t distribution for the log-volatility and then express these two Student-t distributions as a scale mixture of normal (SMN) distributions. Our approach separates the sources of outliers and allows for distinguishing between outliers generated by the return process or by the volatility process, and hence is an improvement over the approach of Choy et al. (2008). In addition, it allows an efficient model implementation using the WinBUGS software. A simulation study is conducted to assess the performance of the proposed approach and its comparison with the approach by Choy et al. (2008). In the empirical study, daily exchange rate returns of the Australian dollar to various currencies and daily stock market index returns of various international stock markets are analysed. Model comparison relies on the Deviance Information Criterion and convergence diagnostic is monitored by Geweke’s convergence test.     

Static and dynamic glass transitions in the 10-state Potts glass: What can Monte Carlo simulations contribute?

The p-state Potts glass with infinite range Gaussian interactions can be solved exactly in the thermodynamic limit and exhibits an unconventional phase behavior if p>4: A dynamical transition from ergodic to non-ergodic behavior at a temperature T_D is followed by a first order transition at T₀<T_D, where a glass order parameter appears discontinuously, although the latent heat is zero. If one assumes that a similar scenario occurs for the structural glass transition as well (though with the singular behavior at T_D rounded off), the p-state Potts glass should be a good test case to develop methods to deal with finite size effects for the static as well as the dynamic transition, and to check what remnants of these unconventional transitions are left in finite sized systems, as they are used in simulations. While it is shown that a sensible extrapolation N→∞ of the simulation results are compatible with the exact results, we find that it would be rather difficult to obtain a correct understanding of the behavior of the system in the thermodynamic limit if only the numerical data would be available.     

Set Membership approximation theory for fast implementation of Model Predictive Control laws

In this paper, the use of Set Membership (SM) methodologies is investigated in the approximation of Model Predictive Control (MPC) laws for linear systems. Such approximated MPC laws are derived from a finite number ν of exact control moves computed off-line. Properties, in terms of guaranteed approximation error, closed-loop stability and performance, are derived assuming only the continuity of the exact predictive control law. These results are achieved by means of two main contributions. At first, it will be shown that if the approximating function enjoys two key properties (i.e. fulfillment of input constraints and explicit evaluation of a bound on the approximation error, which converges to zero as ν increases), then it is possible to guarantee the boundedness of the controlled state trajectories inside a compact set, their convergency to an arbitrary small neighborhood of the origin, and satisfaction of state constraints. Moreover, the guaranteed performance degradation, in terms of maximum state trajectory distance, can be explicitly computed and reduced to an arbitrary small value, by increasing ν. Then, two SM approximations are investigated, both enjoying the above key properties. The first one minimizes the guaranteed approximation error, but its on-line computational time increases with ν. The second one has higher approximation error, but lower on-line computational time which is constant with ν when the off-line computed moves are suitably chosen. The presented approximation techniques can be systematically employed to obtain an efficient MPC implementation for “fast” processes. The effectiveness of the proposed techniques is tested on two numerical examples.     

Approximate dynamic programming-based approaches for input-output data-driven control of nonlinear processes

We propose two approximate dynamic programming (ADP)-based strategies for control of nonlinear processes using input-output data. In the first strategy, which we term ‘J-learning,’ one builds an empirical nonlinear model using closed-loop test data and performs dynamic programming with it to derive an improved control policy. In the second strategy, called ‘Q-learning,’ one tries to learn an improved control policy in a model-less manner. Compared to the conventional model predictive control approach, the new approach offers some practical advantages in using nonlinear empirical models for process control. Besides the potential reduction in the on-line computational burden, it offers a convenient way to control the degree of model extrapolation in the calculation of optimal control moves. One major difficulty associated with using an empirical model within the multi-step predictive control setting is that the model can be excessively extrapolated into regions of the state space where identification data were scarce or nonexistent, leading to performances far worse than predicted by the model. Within the proposed ADP-based strategies, this problem is handled by imposing a penalty term designed on the basis of local data distribution. A CSTR example is provided to illustrate the proposed approaches.     

TaylUR 3, a multivariate arbitrary-order automatic differentiation package for Fortran 95

This new version of TaylUR is based on a completely new core, which now is able to compute the numerical values of all of a complex-valued function's partial derivatives up to an arbitrary order, including mixed partial derivatives.

New version program summary

Program title: TaylURCatalogue identifier: ADXR_v3_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/ADXR_v3_0.htmlProgram obtainable from: CPC Program Library, Queen's University, Belfast, N. IrelandLicensing provisions: GPLv2No. of lines in distributed program, including test data, etc.: 6750No. of bytes in distributed program, including test data, etc.: 19 162Distribution format: tar.gzProgramming language: Fortran 95Computer: Any computer with a conforming Fortran 95 compilerOperating system: Any system with a conforming Fortran 95 compilerClassification: 4.12, 4.14Catalogue identifier of previous version: ADXR_v2_0Journal reference of previous version: Comput. Phys. Comm. 176 (2007) 710Does the new version supersede the previous version?: YesNature of problem: Problems that require potentially high orders of partial derivatives with respect to several variables or derivatives of complex-valued functions, such as e.g. momentum or mass expansions of Feynman diagrams in perturbative QFT, and which previous versions of this TaylUR [1,2] cannot handle due to their lack of support for mixed partial derivatives.Solution method: Arithmetic operators and Fortran intrinsics are overloaded to act correctly on objects of a defined type taylor, which encodes a function along with its first few partial derivatives with respect to the user-defined independent variables. Derivatives of products and composite functions are computed using multivariate forms [3] of Leibniz's rule where ν=(ν₁,…,νd), , , Dνf=^∂|ν|f/(ν₁^∂x₁?νd^∂xd), and μ<ν iff either |μ|<|ν| or |μ|=|ν|,μ₁=ν₁,…,μk=νk,μk+1<νk+1 for some k∈{0,…,d−1}, and of Fàa di Bruno's formula where the sum is over , . An indexed storage system is used to store the higher-order derivative tensors in a one-dimensional array. The relevant indices (k₁,…,ks;λ₁,…,λs) and the weights occurring in the sums in Leibniz's and Fàa di Bruno's formula are precomputed at startup and stored in static arrays for later use.Reasons for new version: The earlier version lacked support for mixed partial derivatives, but a number of projects of interest required them.Summary of revisions: The internal representation of a taylor object has changed to a one-dimensional array which contains the partial derivatives in ascending order, and in lexicographic order of the corresponding multiindex within the same order. The necessary mappings between multiindices and indices into the taylor objects' internal array are computed at startup. To support the change to a genuinely multivariate taylor type, the DERIVATIVE function is now implemented via an interface that accepts both the older format derivative(f,mu,n) and also a new format derivative(f,mu(:))=Dμf that allows access to mixed partial derivatives. Another related extension to the functionality of the module is the HESSIAN function that returns the Hessian matrix of second derivatives of its argument. Since the calculation of all mixed partial derivatives can be very costly, and in many cases only some subset is actually needed, a masking facility has been added. Calling the subroutine DEACTIVATE_DERIVATIVE with a multiindex as an argument will deactivate the calculation of the partial derivative belonging to that multiindex, and of all partial derivatives it can feed into. Similarly, calling the subroutine ACTIVATE_DERIVATIVE will activate the calculation of the partial derivative belonging to its argument, and of all partial derivatives that can feed into it. Moreover, it is possible to turn off the computation of mixed derivatives altogether by setting Diagonal_taylors to .TRUE.. It should be noted that any change of Diagonal_taylors or Taylor_order invalidates all existing taylor objects. To aid the better integration of TaylUR into the HPSrc library [4], routines SET_DERIVATIVE and SET_ALL_DERIVATIVES are provided as a means of manually constructing a taylor object with given derivatives.Restrictions: Memory and CPU time constraints may restrict the number of variables and Taylor expansion order that can be achieved. Loss of numerical accuracy due to cancellation may become an issue at very high orders.Unusual features: These are the same as in previous versions, but are enumerated again here for clarity. The complex conjugation operation assumes all independent variables to be real. The functions REAL and AIMAG do not convert to real type, but return a result of type taylor (with the real/imaginary part of each derivative taken) instead. The user-defined functions VALUE, REALVALUE and IMAGVALUE, which return the value of a taylor object as a complex number, and the real and imaginary part of this value, respectively, as a real number are also provided. Fortran 95 intrinsics that are defined only for arguments of real type (ACOS, AINT, ANINT, ASIN, ATAN, ATAN2, CEILING, DIM, FLOOR, INT, LOG10, MAX, MAXLOC, MAXVAL, MIN, MINLOC, MINVAL, MOD, MODULO, NINT, SIGN) will silently take the real part of taylor-valued arguments unless the module variable Real_args_warn is set to .TRUE., in which case they will return a quiet NaN value (if supported by the compiler) when called with a taylor argument whose imaginary part exceeds the module variable Real_args_tol. In those cases where the derivative of a function becomes undefined at certain points (as for ABS, AINT, ANINT, MAX, MIN, MOD, and MODULO), while the value is well defined, the derivative fields will be filled with quiet NaN values (if supported by the compiler).Additional comments: This version of TaylUR is released under the second version of the GNU General Public License (GPLv2). Therefore anyone is free to use or modify the code for their own calculations. As part of the licensing, it is requested that any publications including results from the use of TaylUR or any modification derived from it cite Refs. [1,2] as well as this paper. Finally, users are also requested to communicate to the author details of such publications, as well as of any bugs found or of required or useful modifications made or desired by them.Running time: The running time of TaylUR operations grows rapidly with both the number of variables and the Taylor expansion order. Judicious use of the masking facility to drop unneeded higher derivatives can lead to significant accelerations, as can activation of the Diagonal_taylors variable whenever mixed partial derivatives are not needed.Acknowledgments: The author thanks Alistair Hart for helpful comments and suggestions. This work is supported by the Deutsche Forschungsgemeinschaft in the SFB/TR 09.References:

[1]

G.M. von Hippel, TaylUR, an arbitrary-order diagonal automatic differentiation package for Fortran 95, Comput. Phys. Comm. 174 (2006) 569.

[2]

G.M. von Hippel, New version announcement for TaylUR, an arbitrary-order diagonal automatic differentiation package for Fortran 95, Comput. Phys. Comm. 176 (2007) 710.

[3]

G.M. Constantine, T.H. Savits, A multivariate Faa di Bruno formula with applications, Trans. Amer. Math. Soc. 348 (2) (1996) 503.

[4]

A. Hart, G.M. von Hippel, R.R. Horgan, E.H. Müller, Automated generation of lattice QCD Feynman rules, Comput. Phys. Comm. 180 (2009) 2698, doi:10.1016/j.cpc.2009.04.021, arXiv:0904.0375.

     

Nonparametric linear tests with multiple events

A generalization of the nonparametric linear rank statistics is presented to handle the two-group comparison with multiple events. For a sample divided into two groups, in which each subject may experience at least two distinct failures, the logrank tests are extended to test the null hypothesis that the vector of the marginal survival distributions of the first group equals that of the second group. Two cases are distinguished depending on whether the null hypothesis does or does not imply the equality of the joint survival functions. In both cases, under the null hypothesis, the asymptotic joint distribution of the vector of the marginal statistics is shown to be Gaussian with covariance matrix consistently estimated using martingale properties. These theoretical results are illustrated by a simulation study and an application on the German Breast Cancer data. An extension to multiple hypotheses testing in multivariate proportional hazards models is also developed.     

Testing for serial independence of panel errors

A test for the serial independence of errors in panel data models is proposed. The test is based on the difference between the joint empirical characteristic function of residuals at different lags and the product of their marginal empirical characteristic functions. The test is nuisance-parameter-free and powerful against any type of pairwise dependence at all lags. A simple random permutation procedure is used to approximate the limit distribution of the test. A Monte Carlo experiment illustrates the finite sample performance of the test, and supports that the test statistic based on the estimated residuals has the same asymptotic distribution as the corresponding statistic based on the unobservable true errors.