Regularized simultaneous model selection in multiple quantiles regression

Simultaneously estimating multiple conditional quantiles is often regarded as a more appropriate regression tool than the usual conditional mean regression for exploring the stochastic relationship between the response and covariates. When multiple quantile regressions are considered, it is of great importance to share strength among them. In this paper, we propose a novel regularization method that explores the similarity among multiple quantile regressions by selecting a common subset of covariates to model multiple conditional quantiles simultaneously. The penalty we employ is a matrix norm that encourages sparsity in a column-wise fashion. We demonstrate the effectiveness of the proposed method using both simulations and an application of gene expression data analysis.     

Design-based estimation for geometric quantiles with application to outlier detection

Geometric quantiles are investigated using data collected from a complex survey. Geometric quantiles are an extension of univariate quantiles in a multivariate set-up that uses the geometry of multivariate data clouds. A very important application of geometric quantiles is the detection of outliers in multivariate data by means of quantile contours. A design-based estimator of geometric quantiles is constructed and used to compute quantile contours in order to detect outliers in both multivariate data and survey sampling set-ups. An algorithm for computing geometric quantile estimates is also developed. Under broad assumptions, the asymptotic variance of the quantile estimator is derived and a consistent variance estimator is proposed. Theoretical results are illustrated with simulated and real data.     

Application of the bootstrap method for estimation of the quantile function

The bootstrap method was used to reduce the sample volume at estimating the quantile function. Accuracy of the quantile sample estimate vs. the distribution of random variable was established analytically. A numerical example of calculation of the quantiles using the proposed bootstrap procedure for the uniform, normal, and Cauchy distributions was considered. An approximate formula for calculation of the quantile of the normal distribution was determined.     

Quantile regression methods with varying-coefficient models for censored data

Considerable intellectual progress has been made to the development of various semiparametric varying-coefficient models over the past ten to fifteen years. An important advantage of these models is that they avoid much of the curse of dimensionality problem as the nonparametric functions are restricted only to some variables. More recently, varying-coefficient methods have been applied to quantile regression modeling, but all previous studies assume that the data are fully observed. The main purpose of this paper is to develop a varying-coefficient approach to the estimation of regression quantiles under random data censoring. We use a weighted inverse probability approach to account for censoring, and propose a majorize–minimize type algorithm to optimize the non-smooth objective function. The asymptotic properties of the proposed estimator of the nonparametric functions are studied, and a resampling method is developed for obtaining the estimator of the sampling variance. An important aspect of our method is that it allows the censoring time to depend on the covariates. Additionally, we show that this varying-coefficient procedure can be further improved when implemented within a composite quantile regression framework. Composite quantile regression has recently gained considerable attention due to its ability to combine information across different quantile functions. We assess the finite sample properties of the proposed procedures in simulated studies. A real data application is also considered.     

Interquantile shrinkage and variable selection in quantile regression

Examination of multiple conditional quantile functions provides a comprehensive view of the relationship between the response and covariates. In situations where quantile slope coefficients share some common features, estimation efficiency and model interpretability can be improved by utilizing such commonality across quantiles. Furthermore, elimination of irrelevant predictors will also aid in estimation and interpretation. These motivations lead to the development of two penalization methods, which can identify the interquantile commonality and nonzero quantile coefficients simultaneously. The developed methods are based on a fused penalty that encourages sparsity of both quantile coefficients and interquantile slope differences. The oracle properties of the proposed penalization methods are established. Through numerical investigations, it is demonstrated that the proposed methods lead to simpler model structure and higher estimation efficiency than the traditional quantile regression estimation.     

Censored depth quantiles

Quantile regression is a wide spread regression technique which allows to model the entire conditional distribution of the response variable. A natural extension to the case of censored observations has been introduced using a reweighting scheme based on the Kaplan-Meier estimator. The same ideas can be applied to depth quantiles. This leads to regression quantiles for censored data which are robust to both outliers in the predictor and the response variable. For their computation, a fast algorithm over a grid of quantile values is proposed. The robustness of the method is shown in a simulation study and on two real data examples.     

Use of a quantile regression based echo state network ensemble for construction of prediction Intervals of gas flow in a blast furnace

The usual huge fluctuations in the blast furnace gas (BFG) generation make the scheduling of the gas system become a difficult problem. Considering that there are high level noises and outliers mixed in original industrial data, a quantile regression-based echo state network ensemble (QR-ESNE) is modeled to construct the prediction intervals (PIs) of the BFG generation. In the process of network training, a linear regression model of the output matrix is reported by the proposed quantile regression to improve the generalization ability. Then, in view of the practical demands on reliability and further improving the prediction accuracy, a bootstrap strategy based on QR-ESN is designed to construct the confidence intervals and the prediction ones via combining with the regression models of various quantiles. To verify the performance of the proposed method, the practical data coming from a steel plant are employed, and the results indicate that the proposed method exhibits high accuracy and reliability for the industrial data. Furthermore, an application software system based on the proposed method is developed and applied to the practice of this plant.     

Asymptotically efficient estimation of the conditional expected shortfall

A procedure for efficient estimation of the trimmed mean of a random variable conditional on a set of covariates is proposed. For concreteness, the focus is on a financial application where the trimmed mean of interest corresponds to the conditional expected shortfall, which is known to be a coherent risk measure. The proposed class of estimators is based on representing the estimator as an integral of the conditional quantile function. Relative to the simple analog estimator that weights all conditional quantiles equally, asymptotic efficiency gains may be attained by giving different weights to the different conditional quantiles while penalizing excessive departures from uniform weighting. The approach presented here allows for either parametric or nonparametric modeling of the conditional quantiles and the weights, but is essentially nonparametric in spirit. The asymptotic properties of the proposed class of estimators are established. Their finite sample properties are illustrated through a set of Monte Carlo experiments and an empirical application¹.     

Quantile estimation in two-phase sampling

The estimation of quantiles in two-phase sampling with arbitrary sampling design in each of the two phases is investigated. Several ratio and exponentiation type estimators that provide the optimum estimate of a quantile based on an optimum exponent α are proposed. Properties of these estimators are studied under large sample size approximation and the use of double sampling for stratification to estimate quantiles can also be seen. The real performance of these estimators will be evaluated for the three quartiles on the basis of data from two real populations using different sampling designs. The simulation study shows that proposed estimators can be very satisfactory in terms of relative bias and efficiency.     

Bayesian causal effects in quantiles: Accounting for heteroscedasticity

Testing for Granger non-causality over varying quantile levels could be used to measure and infer dynamic linkages, enabling the identification of quantiles for which causality is relevant, or not. However, dynamic quantiles in financial application settings are clearly affected by heteroscedasticity, as well as the exogenous and endogenous variables under consideration. GARCH-type dynamics are added to the standard quantile regression model, so as to more robustly examine quantile causal relations between dynamic variables. An adaptive Bayesian Markov chain Monte Carlo scheme, exploiting the link between quantile regression and the skewed-Laplace distribution, is designed for estimation and inference of the quantile causal relations, simultaneously estimating and accounting for heteroscedasticity. Dynamic quantile linkages for the international stock markets in Taiwan and Hong Kong are considered over a range of quantile levels. Specifically, the hypothesis that these stock returns are Granger-caused by the US market and/or the Japanese market is examined. The US market is found to significantly and positively Granger-cause both markets at all quantile levels, while the Japanese market effect was also significant at most quantile levels, but with weaker effects.     

Quantile autoregression neural network model with applications to evaluating value at risk

We develop a new quantile autoregression neural network (QARNN) model based on an artificial neural network architecture. The proposed QARNN model is flexible and can be used to explore potential nonlinear relationships among quantiles in time series data. By optimizing an approximate error function and standard gradient based optimization algorithms, QARNN outputs conditional quantile functions recursively. The utility of our new model is illustrated by Monte Carlo simulation studies and empirical analyses of three real stock indices from the Hong Kong Hang Seng Index (HSI), the US S&P500 Index (S&P500) and the Financial Times Stock Exchange 100 Index (FTSE100).     

Fundamentals of the linearization method for quantile analysis with small random parameters

Consideration was given to determination of the quantiles of the distribution of a nonlinear loss function of small random parameters. It was suggested to construct the quantile estimates using a linearized model where the problem under study comes to that of generalized linear programming. Precision of these estimates was examined.     

Estimating steady-state distributions via simulation-generated histograms

This paper discusses a unified approach for estimating, via a histogram, the steady-state distribution of a stochastic process observed by simulation. The quasi-independent   (QI) procedure increases the simulation run length progressively until a certain number of essentially independent and identically distributed samples are obtained. It is known that order-statistics quantile estimators are asymptotically unbiased when the output sequences satisfy certain conditions. We compute sample quantiles at certain grid points and use Lagrange interpolation to estimate any p$p$ quantile. Our quantile estimators satisfy a proportional-precision requirement at the first phase, and a relative- or absolute-precision requirement at the second phase. An experimental performance evaluation demonstrates the validity of using the QI procedure to estimate quantiles and construct a histogram to estimate the steady-state distribution.     

A quantile estimation for massive data with generalized Pareto distribution

This paper proposes a new method of estimating extreme quantiles of heavy-tailed distributions for massive data. The method utilizes the Peak Over Threshold (POT) method with generalized Pareto distribution (GPD) that is commonly used to estimate extreme quantiles and the parameter estimation of GPD using the empirical distribution function (EDF) and nonlinear least squares (NLS). We first estimate the parameters of GPD using EDF and NLS and then, estimate multiple high quantiles for massive data based on observations over a certain threshold value using the conventional POT. The simulation results demonstrate that our parameter estimation method has a smaller Mean square error (MSE) than other common methods when the shape parameter of GPD is at least 0. The estimated quantiles also show the best performance in terms of root MSE (RMSE) and absolute relative bias (ARB) for heavy-tailed distributions.     

Dual Stochastic Dominance and Quantile Risk Measures

Following the seminal work by Markowitz, the portfolio selection problem is usually modeled as a bicriteria optimization problem where a reasonable trade–off between expected rate of return and risk is sought. In the classical Markowitz model, the risk is measured with variance. Several other risk measures have been later considered thus creating the entire family of mean–risk (Markowitz type) models. In this paper, we analyze mean–risk models using quantiles and tail characteristics of the distribution. Value at risk (VAR), defined as the maximum loss at a specified confidence level, is a widely used quantile risk measure. The corresponding second order quantile measure, called the worst conditional expectation or Tail VAR, represents the mean shortfall at a specified confidence level. It has more attractive theoretical properties and it leads to LP solvable portfolio optimization models in the case of discrete random variables, i.e., in the case of returns defined by their realizations under the specified scenarios. We show that the mean–risk models using the worst conditional expectation or some of its extensions are in harmony with the stochastic dominance order. For this purpose, we exploit duality relations of convex analysis to develop the quantile model of stochastic dominance for general distributions.     

Quantile map: Simultaneous visualization of patterns in many distributions with application to tandem mass spectrometry

High-throughput experiments have become more and more prevalent in biomedical research. The resulting high-dimensional data have brought new challenges. Effective data reduction, summarization and visualization are important keys to initial exploration in data mining. In this paper, we introduce a visualization tool, namely a quantile map, to present information contained in a probabilistic distribution. We demonstrate its use as an effective visual analysis tool through the application of a tandem mass spectrometry data set. Information of quantiles of a distribution is presented in gradient colors by concentric doughnuts. The width of the doughnuts is proportional to the Fisher information of the distribution to present unbiased visualization effect. A parametric empirical Bayes (PEB) approach is shown to improve the simple maximum likelihood estimate (MLE) approach when estimating the Fisher information. In the motivating example from tandem mass spectrometry data, multiple probabilistic distributions are to be displayed in two-dimensional grids. A hierarchical clustering to reorder rows and columns and a gradient color selection from a Hue-Chroma-Luminance model, similar to that commonly applied in heatmaps of microarray analysis, are adopted to improve the visualization. Both simulations and the motivating example show superior performance of the quantile map in summarization and visualization of such high-throughput data sets.     

Estimating quantiles under sampling on two occasions with arbitrary sample designs

A practical problem related to the estimation of quantiles in double sampling with arbitrary sampling designs in each of the two phases is investigated. In practice, this scheme is commonly used for official surveys, in which quantile estimation is often required when the investigation deals with variables such as income or expenditure. A class of estimators for quantiles is proposed and some important properties, such as asymptotic unbiasedness and asymptotic variance, are established. The optimal estimator, in the sense of minimizing the asymptotic variance, is also presented. The proposed class contains several known types of estimators, such as ratio and regression estimators, which are of practical application and are therefore derived. Assuming several populations, the proposed estimators are compared with the direct estimator via an empirical study. Results show that a gain in efficiency can be obtained.     

Estimating quantiles under sampling on two occasions with arbitrary sample designs

A practical problem related to the estimation of quantiles in double sampling with arbitrary sampling designs in each of the two phases is investigated. In practice, this scheme is commonly used for official surveys, in which quantile estimation is often required when the investigation deals with variables such as income or expenditure. A class of estimators for quantiles is proposed and some important properties, such as asymptotic unbiasedness and asymptotic variance, are established. The optimal estimator, in the sense of minimizing the asymptotic variance, is also presented. The proposed class contains several known types of estimators, such as ratio and regression estimators, which are of practical application and are therefore derived. Assuming several populations, the proposed estimators are compared with the direct estimator via an empirical study. Results show that a gain in efficiency can be obtained.     

Aggregation of classifiers ensemble using local discriminatory power and quantiles

The paper presents a new approach to the dynamic classifier selection in an ensemble by applying the best suited classifier for the particular testing sample. It is based on the area under curve (AUC) of the receiver operating characteristic (ROC) of each classifier. To allow application of different types of classifiers in an ensemble and to reduce the influence of outliers, the quantile representation of the signals is used. The quantiles divide the ordered data into essentially equal-sized data subsets providing approximately uniform distribution of [0–1] support for each data point. In this way the recognition problem is less sensitive to the outliers, scales and noise contained in the input attributes. The numerical results presented for the chosen benchmark data-mining sets and for the data-set of images representing melanoma and non-melanoma skin lesions have shown high efficiency of the proposed approach and superiority to the existing methods.