Likelihood-free Bayesian estimation of multivariate quantile distributions

In this paper, we present new multivariate quantile distributions and utilise likelihood-free Bayesian algorithms for inferring the parameters. In particular, we apply a sequential Monte Carlo (SMC) algorithm that is adaptive in nature and requires very little tuning compared with other approximate Bayesian computation algorithms. Furthermore, we present a framework for the development of multivariate quantile distributions based on a copula. We consider bivariate and time series extensions of the g-and-k distribution under this framework, and develop an efficient component-wise updating scheme free of likelihood functions to be used within the SMC algorithm. In addition, we trial the set of octiles as summary statistics as well as functions of these that form robust measures of location, scale, skewness and kurtosis. We show that these modifications lead to reasonably precise inferences that are more closely comparable to computationally intensive likelihood-based inference. We apply the quantile distributions and algorithms to simulated data and an example involving daily exchange rate returns.     

Worst case control of uncertain jumping systems with multi-state and input delay information

In this paper, the problem of worst case (also called H_∞) Control for a class of uncertain systems with Markovian jump parameters and multiple delays in the state and input is investigated. The jumping parameters are modelled as a continuous-time, discrete-state Markov process and the parametric uncertainties are assumed to be real, time-varying and norm-bounded that appear in the state, input and delayed-state matrices. The time-delay factors are unknowns and time-varying with known bounds. Complete results for instantaneous and delayed state feedback control designs are developed which guarantee the weak-delay dependent stochastic stability with a prescribed H_∞-performance. The solutions are provided in terms of a finite set of coupled linear matrix inequalities (LMIs). Application of the developed theory to a typical example has been presented.     

A model reference approach to safe controller changes in iterative identification and control

A controller change from a current controller which stabilises the plant to a new controller, designed on the basis of an approximate model of the plant and with guaranteed bounds on the stability properties of the true closed loop, is called a safe controller change. In this paper, we present a model reference approach to the determination of safe controller changes on the basis of approximate closed loop models of the plant and robust stability results in the ν-gap.     

Bayesian skew selection for multivariate models

We develop a Bayesian approach for the selection of skew in multivariate skew t distributions constructed through hidden conditioning in the manners suggested by either Azzalini and Capitanio (2003) or Sahu et al. (2003). We show that the skew coefficients for each margin are the same for the standardized versions of both distributions. We introduce binary indicators to denote whether there is symmetry, or skew, in each dimension. We adopt a proper beta prior on each non-zero skew coefficient, and derive the corresponding prior on the skew parameters. In both distributions we show that as the degrees of freedom increases, the prior smoothly bounds the non-zero skew parameters away from zero and identifies the posterior. We estimate the model using Markov chain Monte Carlo (MCMC) methods by exploiting the conditionally Gaussian representation of the skew t distributions. This allows for the search through the posterior space of all possible combinations of skew and symmetry in each dimension. We show that the proposed method works well in a simulation setting, and employ it in two multivariate econometric examples. The first involves the modeling of foreign exchange rates and the second is a vector autoregression for intra-day electricity spot prices. The approach selects skew along the original coordinates of the data, which proves insightful in both examples.     

Obtaining more information from conjoint experiments by best-worst choices

Conjoint choice experiments elicit individuals’ preferences for the attributes of a good by asking respondents to indicate repeatedly their most preferred alternative in a number of choice sets. However, conjoint choice experiments can be used to obtain more information than that revealed by the individuals’ single best choices. A way to obtain extra information is by means of best-worst choice experiments in which respondents are asked to indicate not only their most preferred alternative but also their least preferred one in each choice set. To create D-optimal designs for these experiments, an expression for the Fisher information matrix for the maximum-difference model is developed. Semi-Bayesian D-optimal best-worst choice designs are derived and compared with commonly used design strategies in marketing in terms of the D-optimality criterion and prediction accuracy. Finally, it is shown that best-worst choice experiments yield considerably more information than choice experiments.     

Bayesian analysis of semiparametric reproductive dispersion mixed-effects models

Semiparametric reproductive dispersion mixed-effects model (SPRDMM) is an extension of the reproductive dispersion model and the semiparametric mixed model, and it includes many commonly encountered models as its special cases. A Bayesian procedure is developed for analyzing SPRDMMs on the basis of P-spline estimates of nonparametric components. A hybrid algorithm combining the Gibbs sampler and the Metropolis-Hastings algorithm is used to simultaneously obtain the Bayesian estimates of unknown parameters, smoothing function and random effects, as well as their standard error estimates. The Bayes factor for model comparison is employed to select better approximation of the smoothing function via path sampling. Several simulation studies and a real example are used to illustrate the proposed methodologies.     

A parametric bootstrap approach for ANOVA with unequal variances: Fixed and random models

This article is about testing the equality of several normal means when the variances are unknown and arbitrary, i.e., the set up of the one-way ANOVA. Even though several tests are available in the literature, none of them perform well in terms of Type I error probability under various sample size and parameter combinations. In fact, Type I errors can be highly inflated for some of the commonly used tests; a serious issue that appears to have been overlooked. We propose a parametric bootstrap (PB) approach and compare it with three existing location-scale invariant tests—the Welch test, the James test and the generalized F (GF) test. The Type I error rates and powers of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test is the best among the four tests with respect to Type I error rates. The PB test performs very satisfactorily even for small samples while the Welch test and the GF test exhibit poor Type I error properties when the sample sizes are small and/or the number of means to be compared is moderate to large. The James test performs better than the Welch test and the GF test. It is also noted that the same tests can be used to test the significance of the random effect variance component in a one-way random model under unequal error variances. Such models are widely used to analyze data from inter-laboratory studies. The methods are illustrated using some examples.     

An LMI approach to minimum sensitivity analysis with application to fault detection

This paper systematically studies the minimum input sensitivity analysis problem. The lowest level of sensitivity of system outputs to system inputs is defined as an H_- index. A full characterization of the H_- index is given, first, in terms of matrix equalities and inequalities, and then in terms of linear matrix inequalities (LMIs), as a dual of the Bounded Real Lemma. A related problem of input observability is also studied, with new necessary and sufficient conditions given, which are necessary for a fault detection system to have a nonzero worst-case fault sensitivity. The above results are applied to the problem of fault detection filter analysis, with numerical examples given to show the effectiveness of the proposed approaches.     

Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control

In this paper, the optimal strategies for discrete-time linear system quadratic zero-sum games related to the H-infinity optimal control problem are solved in forward time without knowing the system dynamical matrices. The idea is to solve for an action dependent value function Q(x,u,w) of the zero-sum game instead of solving for the state dependent value function V(x) which satisfies a corresponding game algebraic Riccati equation (GARE). Since the state and actions spaces are continuous, two action networks and one critic network are used that are adaptively tuned in forward time using adaptive critic methods. The result is a Q-learning approximate dynamic programming (ADP) model-free approach that solves the zero-sum game forward in time. It is shown that the critic converges to the game value function and the action networks converge to the Nash equilibrium of the game. Proofs of convergence of the algorithm are shown. It is proven that the algorithm ends up to be a model-free iterative algorithm to solve the GARE of the linear quadratic discrete-time zero-sum game. The effectiveness of this method is shown by performing an H-infinity control autopilot design for an F-16 aircraft.     

A new approach to robust guaranteed cost control for uncertain multimodeling systems

The guaranteed cost control problem for multimodeling systems with norm bounded uncertainty is investigated. The main contribution in this paper is that a new ?-independent controller is derived by solving the reduced-order slow and fast algebraic Riccati equations (AREs) whose dimension is smaller than the dimension of full-order multiparameter algebraic Riccati equation (MARE). It is shown that if these AREs have a positive definite stabilizing solution then the closed-loop system is quadratically stable and has the cost bound.     

Robust model selection with flexible trimming

The forward search provides data-driven flexible trimming of a C_p statistic for the choice of regression models that reveals the effect of outliers on model selection. An informed robust model choice follows. Even in small samples, the statistic has a null distribution indistinguishable from an F distribution. Limits on acceptable values of the C_p statistic follow. Two examples of widely differing size are discussed. A powerful graphical tool is the generalized candlestick plot, which summarizes the information on all forward searches and on the choice of models. A comparison is made with the use of M-estimation in robust model choice.     

Bootstrap approach to the multi-sample test of means with imprecise data

A bootstrap approach to the multi-sample test of means for imprecisely valued sample data is introduced. For this purpose imprecise data are modelled in terms of fuzzy values. Populations are identified with fuzzy-valued random elements, often referred to in the literature as fuzzy random variables. An example illustrates the use of the suggested method. Finally, the adequacy of the bootstrap approach to test the multi-sample hypothesis of means is discussed through a simulation comparative study.     

A distributed controller approach for delay-independent stability of networked control systems

This article introduces a novel distributed controller approach for networked control systems (NCS) to achieve finite gain L₂ stability independent of constant time delay. The proposed approach represents a generalization of the well-known scattering transformation which applies for passive systems only. The main results of this article are (a) a sufficient stability condition for general multi-input-multi-output (MIMO) input-feedforward-output-feedback-passive (IF-OFP) nonlinear systems and (b) a necessary and sufficient stability condition for linear time-invariant (LTI) single-input-single-output (SISO) systems. The performance advantages of the proposed approach are reduced sensitivity to time delay and improved steady state error compared to alternative known delay-independent small gain type approaches. Simulations validate the proposed approach.     

State feedback control of continuous-time T-S fuzzy systems via switched fuzzy controllers

This paper studies the problem of state feedback control of continuous-time T-S fuzzy systems. Switched fuzzy controllers are exploited in the control design, which are switched based on the values of membership functions, and the control scheme is an extension of the parallel distributed compensation (PDC) scheme. Sufficient conditions for designing switched state feedback controllers are obtained with meeting an H_∞ norm bound requirement and quadratic D stability constraints. It is shown that the new control design method provides less conservative results than the corresponding ones via the parallel distributed compensation (PDC) scheme. A numerical example is given to illustrate the effectiveness of the proposed method.     

A subband coding approach to control under limited data rates and message losses

For the design of networked control systems, we employ a subband coding technique to efficiently use the available data rate. Such coding schemes have widely been used in signal processing for data compression. We take an approach suitable from the control perspective in that frequency characteristics of a controller are directly used. In particular, we propose a three-step design procedure for the overall controller having the following features: On the coder side, it uses a controller consisting of a filter bank whose outputs are subband signals of the control input. On the decoder side is another filter bank for reconstruction of the subband signals. This decoder is capable to account for random message losses that occur in communication and is designed via an H^∞-type method. For the quantizers in the coder, an efficient bit allocation scheme is also developed.     

Robust observers for neutral jumping systems with uncertain information

In this paper, the problem of designing observer for a class of uncertain neutral systems. The uncertainties are parametric and norm-bounded. Both robust observation and robust H_∞ observation methods are developed by using linear state-delayed observers. In case of robust observation, sufficient conditions are established for asymptotic stability of the system, which is independent of time delay. The results are then extended to robust H_∞ observation which renders the augmented system asymptotically stable independent of delay with a guaranteed performance measure. Furthermore, a memoryless state-estimate feedback is designed to stabilize the closed-loop neutral system. In all cases, the gain matrices are determined by linear matrix inequality approach. Two numerical examples are presented to illustrate the validity of the theoretical results.     

Smooth semiparametric and nonparametric Bayesian estimation of bivariate densities from bivariate histogram data

Penalized B-splines combined with the composite link model are used to estimate a bivariate density from a histogram with wide bins. The goals are multiple: they include the visualization of the dependence between the two variates, but also the estimation of derived quantities like Kendall’s tau, conditional moments and quantiles. Two strategies are proposed: the first one is semiparametric with flexible margins modeled using B-splines and a parametric copula for the dependence structure; the second one is nonparametric and is based on Kronecker products of the marginal B-spline bases. Frequentist and Bayesian estimations are described. A large simulation study quantifies the performances of the two methods under different dependence structures and for varying strengths of dependence, sample sizes and amounts of grouping. It suggests that Schwarz’s BIC is a good tool for classifying the competing models. The density estimates are used to evaluate conditional quantiles in two applications in social and in medical sciences.     

A simple and efficient method for variable ranking according to their usefulness for learning

The selection of a subset of input variables is often based on the previous construction of a ranking to order the variables according to a given criterion of relevancy. The objective is then to linearize the search, estimating the quality of subsets containing the topmost ranked variables. An algorithm devised to rank input variables according to their usefulness in the context of a learning task is presented. This algorithm is the result of a combination of simple and classical techniques, like correlation and orthogonalization, which allow the construction of a fast algorithm that also deals explicitly with redundancy. Additionally, the proposed ranker is endowed with a simple polynomial expansion of the input variables to cope with nonlinear problems. The comparison with some state-of-the-art rankers showed that this combination of simple components is able to yield high-quality rankings of input variables. The experimental validation is made on a wide range of artificial data sets and the quality of the rankings is assessed using a ROC-inspired setting, to avoid biased estimations due to any particular learning algorithm.     

Threshold-based clustering with merging and regularization in application to network intrusion detection

Signature-based intrusion detection systems look for known, suspicious patterns in the input data. In this paper we explore compression of labeled empirical data using threshold-based clustering with regularization. The main target of clustering is to compress training dataset to the limited number of signatures, and to minimize the number of comparisons that are necessary to determine the status of the input event as a result. Essentially, the process of clustering includes merging of the clusters which are close enough. As a consequence, we will reduce original dataset to the limited number of labeled centroids. In a complex with k-nearest-neighbor (kNN) method, this set of centroids may be used as a multi-class classifier. The experiments on the KDD-99 intrusion detection dataset have confirmed effectiveness of the above procedure.     

Parameterized complexity of control problems in Maximin election

Elections are a central model in a variety of areas. This paper studies parameterized computational complexity of five control problems in the Maximin election. We obtain the following results: constructive control by adding candidates is W[2]-hard with respect to the parameter “number of added candidates”; both constructive and destructive control by adding/deleting voters are W[1]-hard with respect to the parameter “number of added/deleted voters”.