Random effects mixture models for clustering electrical load series

For purposes such as rate setting and long‐term capacity planning, electrical utility companies are interested in dividing their customers into homogeneous groups or clusters in terms of the customers’ electricity demand profiles. Such demand profiles are typically represented by load series, long time series of daily or even hourly rates of energy consumption of individual customers. The high dimension and time series nature inherent in the load series render existing methods of clustering analysis ineffective. To handle the high dimension and to take advantage of the time‐series nature of load series, we introduce a class of mixture models for time series, the random effects mixture models, which are particularly useful for clustering the load series. The random effects mixture models are based on a hierarchical model for individual components. They employ highly flexible antedependence models to effectively capture the time‐series characteristics of the covariance of the load series. We present details on the construction of such mixture models and discuss a special Expectation‐maximization (EM) algorithm for their computation. We also apply these models to cluster the data set which had motivated this research, a set of 923 load series from BC Hydro, a crown utility company in British Columbia, Canada.     

A p‐Order signed integer‐valued autoregressive (SINAR(p)) model

In this article, we propose an extension of integer‐valued autoregressive INAR models. Using a signed version of the thinning operator, we define a larger class of ‐valued processes, called SINAR, which can have positive as well as negative correlations. Using a Markov chain method, conditions for stationarity and the existence of moments are investigated. In particular, it is shown that the autocorrelation function of any real‐valued AR process can be recovered with a SINAR process, which improves INAR modeling.     

Estimation for non‐negative time series with heavy‐tail innovations

For moving average processes where the coefficients are non‐negative and the innovations are positive random variables with a regularly varying tail at infinity, we provide estimates for the coefficients based on the ratio of two sample values chosen with respect to an extreme value criteria. We then apply this result to obtain estimates for the parameters of non‐negative ARMA models. Weak convergence results for the joint distribution of our estimates are established and a simulation study is provided to examine the small sample size behaviour of these estimates.     

A negative binomial integer‐valued GARCH model

This article discusses the modelling of integer‐valued time series with overdispersion and potential extreme observations. For the problem, a negative binomial INGARCH model, a generalization of the Poisson INGARCH model, is proposed and stationarity conditions are given as well as the autocorrelation function. For estimation, we present three approaches with the focus on the maximum likelihood approach. Some results from numerical studies are presented and indicate that the proposed methodology performs better than the Poisson and double Poisson model‐based methods.     

A test for second‐order stationarity of a time series based on the discrete Fourier transform

We consider a zero mean discrete time series, and define its discrete Fourier transform (DFT) at the canonical frequencies. It can be shown that the DFT is asymptotically uncorrelated at the canonical frequencies if and only if the time series is second‐order stationary. Exploiting this important property, we construct a Portmanteau type test statistic for testing stationarity of the time series. It is shown that under the null of stationarity, the test statistic has approximately a chi‐square distribution. To examine the power of the test statistic, the asymptotic distribution under the locally stationary alternative is established. It is shown to be a generalized non‐central chi‐square, where the non‐centrality parameter measures the deviation from stationarity. The test is illustrated with simulations, where is it shown to have good power.     

Parameter Estimation and Subset Selection for Separable lower Triangular Bilinear Models

Abstract. Parameter estimation and subset selection for separable lower triangular bilinear (SLTBL) models are considered. Under a flat prior, we present an expectation–maximization (EM) algorithm to obtain the maximum likelihood estimation. Furthermore, two sub‐procedures are designed to select the best subset model after an initial fitting. Example with two simulated and one real data set illustrate the feasibility and validity of the proposed methods.     

On the Spectral Density of the Wavelet Coefficients of Long‐Memory Time Series with Application to the Log‐Regression Estimation of the Memory Parameter

Abstract. In recent years, methods to estimate the memory parameter using wavelet analysis have gained popularity in many areas of science. Despite its widespread use, a rigorous semi‐parametric asymptotic theory, comparable with the one developed for Fourier methods, is still lacking. In this article, we adapt to the wavelet setting, the classical semi‐parametric framework introduced by Robinson and his co‐authors for estimating the memory parameter of a (possibly) non‐stationary process. Our results apply to a class of wavelets with bounded supports, which include but are not limited to Daubechies wavelets. We derive an explicit expression of the spectral density of the wavelet coefficients and show that it can be approximated, at large scales, by the spectral density of the continuous‐time wavelet coefficients of fractional Brownian motion. We derive an explicit bound for the difference between the spectral densities. As an application, we obtain minimax upper bounds for the log‐scale regression estimator of the memory parameter for a Gaussian process and we derive an explicit expression of its asymptotic variance.     

A note on the mixture transition distribution and hidden Markov models

We discuss an interpretation of the mixture transition distribution (MTD) for discrete‐valued time series which is based on a sequence of independent latent variables which are occasion‐specific. We show that, by assuming that this latent process follows a first order Markov Chain, MTD can be generalized in a sensible way. A class of models results which also includes the hidden Markov model (HMM). For these models we outline an EM algorithm for the maximum likelihood estimation which exploits recursions developed within the HMM literature. As an illustration, we provide an example based on the analysis of stock market data referred to different American countries.     

Wavelet‐Based Tests for Comparing Two Time Series with Unequal Lengths

Test procedures for assessing whether two stationary and independent time series with unequal lengths have the same spectral density (or same auto‐covariance function) are investigated. A new test statistic is proposed based on the wavelet transform. It relies on empirical wavelet coefficients of the logarithm of two spectral densities' ratio. Under the null hypothesis that two spectral densities are the same, the asymptotic normal distribution of the empirical wavelet coeffcients is derived. Furthermore, these empirical wavelet coefficients are asymptotically uncorrelated. A test statistic is proposed based on these results. The performance of the new test statistic is compared to several recent test statistics, with respect to their exact levels and powers. Simulation studies show that our proposed test is very comparable to the current test statistics in most cases. The main advantage of our proposed test statistic is that it is constructed very simply and is easy to implement.     

Stationary bootstrapping for non‐parametric estimator of nonlinear autoregressive model

We consider stationary bootstrap approximation of the non‐parametric kernel estimator in a general kth‐order nonlinear autoregressive model under the conditions ensuring that the nonlinear autoregressive process is a geometrically Harris ergodic stationary Markov process. We show that the stationary bootstrap procedure properly estimates the distribution of the non‐parametric kernel estimator. A simulation study is provided to illustrate the theory and to construct confidence intervals, which compares the proposed method favorably with some other bootstrap methods.     

Estimation in Random Coefficient Autoregressive Models

Abstract. We propose the quasi‐maximum likelihood method to estimate the parameters of an RCA(1) process, i.e. a random coefficient autoregressive time series of order 1. The strong consistency and the asymptotic normality of the estimators are derived under optimal conditions.     

Central limit theorems for nonparametric estimators with real‐time random variables

In this article, asymptotic theories for nonparametric methods are studied when they are applied to real‐time data. In particular, we derive central limit theorems for nonparametric density and regression estimators. For this we formally introduce a sequence of real‐time random variables indexed by a parameter related to fine gridding of time domain (or fine discretization). Our results show that the impact of fine gridding is greater in the density estimation case in the sense that strong dependence due to fine gridding severely affects the major strength of nonparametric density estimator (or its data‐adaptive property). In addition, we discuss some issues about nonparametric regression model with fine gridding of time domain.     

On residual empirical processes of GARCH‐SM models: application to conditional symmetry tests

Abstract. Considering the generalized autoregressive conditionally heteroskedastic with stochastic mean (GARCH‐SM) model, we establish in this article the consistency and the weak representation of a functional of its residual empirical process. Based on this result, a symmetry test for GARCH‐SM model is developed. Simulations are given to show the asymptotic behaviour and normality of the test statistic.     

Maximum likelihood estimation for nearly non‐stationary stable autoregressive processes

The maximum likelihood estimate (MLE) of the autoregressive coefficient of a near‐unit root autoregressive process Y_t = ρ_nY_t?1 + ?_t with α‐stable noise {?_t} is studied in this paper. Herein ρ_n = 1 ? γ/n, γ ≥ 0 is a constant, Y₀ is a fixed random variable and ε_t is an α‐stable random variable with characteristic function φ(t,θ) for some parameter θ. It is shown that when 0 < α < 1 or α > 1 and E?₁ = 0, the limit distribution of the MLE of ρ_n and θ are mixtures of a stable process and Gaussian processes. On the other hand, when α > 1 and E?₁ ≠ 0, the limit distribution of the MLE of ρ_n and θ are normal. A Monte Carlo simulation reveals that the MLE performs better than the usual least squares procedures, particularly for the case when the tail index α is less than 1.     

Quantifying the uncertainty introduced by discretization and time‐averaging in two‐fluid model predictions

The two‐fluid model (TFM) has become a tool for the design and troubleshooting of industrial fluidized bed reactors. To use TFM for scale up with confidence, the uncertainty in its predictions must be quantified. Here, we study two sources of uncertainty: discretization and time‐averaging. First, we show that successive grid refinement may not yield grid‐independent transient quantities, including cross‐section–averaged quantities. Successive grid refinement would yield grid‐independent time‐averaged quantities on sufficiently fine grids. Then a Richardson extrapolation can be used to estimate the discretization error, and the grid convergence index gives an estimate of the uncertainty. Richardson extrapolation may not work for industrial‐scale simulations that use coarse grids. We present an alternative method for coarse grids and assess its ability to estimate the discretization error. Second, we assess two methods (autocorrelation and binning) and find that the autocorrelation method is more reliable for estimating the uncertainty introduced by time‐averaging TFM data. © 2017 American Institute of Chemical Engineers AIChE J, 63: 5343–5360, 2017     

Estimation of regression and dynamic dependence paremeters for non‐stationary multinomial time series

In a time‐series regression setup, multinomial responses along with time dependent observable covariates are usually modelled by certain suitable dynamic multinomial logistic probabilities. Frequently, the time‐dependent covariates are treated as a realization of an exogenous random process and one is interested in the estimation of both the regression and the dynamic dependence parameters conditional on this realization of the covariate process. There exists a partial likelihood estimation approach able to deal with the general dependence structures arising from the influence of both past covariates and past multinomial responses on the covariates at a given time by sequentially conditioning on the history of the joint process (response and covariates), but it provides standard errors for the estimators based on the observed information matrix, because such a matrix happens to be the Fisher information matrix obtained by conditioning on the whole history of the joint process. This limitation of the partial likelihood approach holds even if the covariate history is not influeced by lagged response outcomes. In this article, a general formulation of the auto‐covariance structure of a multinomial time series is presented and used to derive an explicit expression for the Fisher information matrix conditional on the covariate history, providing the possibility of computing the variance of the maximum likelihood estimators given a realization of the covariate process for the multinomial‐logistic model. The difference between the standard errors of the parameter estimators under these two conditioning schemes (covariates Vs. joint history) is illustrated through an intensive simulation study based on the premise of an exogenous covariate process.     

Statistical tests for a single change in mean against long‐range dependence

Statistical tests are introduced for distinguishing between short‐range dependent time series with a single change in mean, and long‐range dependent time series, with the former making the null hypothesis. The tests are based on estimation of the self‐similarity parameter after removing the change in mean from the series. The focus is on the GPH (Geweke and Porter‐Hudak, 1983) and local Whittle estimation methods in the spectral domain. Theoretical properties of the resulting estimators are established when testing for a single change in mean, and small sample properties of the tests are examined in simulations. The introduced tests improve on the BHKS ( Berkes et al., 2006 ) test which is the only other available test for the considered problem. It is argued that the BHKS test has a low power against long‐range dependence alternatives and that this happens because the BHKS test statistic involves estimation of the long‐run variance. The BHKS test could be improved readily by considering its R/S‐like regression version which estimates the self‐similarity parameter and which does not involve the long‐run variance. Yet better alternatives are to use more powerful estimation methods (such as GPH or local Whittle) and lead to the tests introduced here.     

Asymptotic self‐similarity and wavelet estimation for long‐range dependent fractional autoregressive integrated moving average time series with stable innovations

Abstract. Methods for parameter estimation in the presence of long‐range dependence and heavy tails are scarce. Fractional autoregressive integrated moving average (FARIMA) time series for positive values of the fractional differencing exponent d can be used to model long‐range dependence in the case of heavy‐tailed distributions. In this paper, we focus on the estimation of the Hurst parameter H = d + 1/α for long‐range dependent FARIMA time series with symmetric α‐stable (1 < α < 2) innovations. We establish the consistency and the asymptotic normality of two types of wavelet estimators of the parameter H. We do so by exploiting the fact that the integrated series is asymptotically self‐similar with parameter H. When the parameter α is known, we also obtain consistent and asymptotically normal estimators for the fractional differencing exponent d = H ? 1/α. Our results hold for a larger class of causal linear processes with stable symmetric innovations. As the wavelet‐based estimation method used here is semi‐parametric, it allows for a more robust treatment of long‐range dependent data than parametric methods.     

Economic model predictive control of nonlinear time‐delay systems: Closed‐loop stability and delay compensation

Closed‐loop stability of nonlinear time‐delay systems under Lyapunov‐based economic model predictive control (LEMPC) is considered. LEMPC is initially formulated with an ordinary differential equation model and is designed on the basis of an explicit stabilizing control law. To address closed‐loop stability under LEMPC, first, we consider the stability properties of the sampled‐data system resulting from the nonlinear continuous‐time delay system with state and input delay under a sample‐and‐hold implementation of the explicit controller. The steady‐state of this sampled‐data closed‐loop system is shown to be practically stable. Second, conditions such that closed‐loop stability, in the sense of boundedness of the closed‐loop state, under LEMPC are derived. A chemical process example is used to demonstrate that indeed closed‐loop stability is maintained under LEMPC for sufficiently small time‐delays. To cope with performance degradation owing to the effect of input delay, a predictor feedback LEMPC methodology is also proposed. The predictor feedback LEMPC design employs a predictor to compute a prediction of the state after the input delay period and an LEMPC scheme that is formulated with a differential difference equation (DDE) model, which describes the time‐delay system, initialized with the predicted state. The predictor feedback LEMPC is also applied to the chemical process example and yields improved closed‐loop stability and economic performance properties. © 2015 American Institute of Chemical Engineers AIChE J, 61: 4152–4165, 2015     

On the properties of the periodogram of a stationary long‐memory process over different epochs with applications

This article studies the asymptotic properties of the discrete Fourier transforms (DFT) and the periodogram of a stationary long‐memory time series over different epochs. The main theoretical result is a novel bound for the covariance of the DFT ordinates evaluated on two distinct epochs, which depends explicitly on the Fourier frequencies and the gap between the epochs. This result is then applied to obtain the limiting distribution of some nonlinear functions of the periodogram over different epochs, under the additional assumption of gaussianity. We then apply this result to construct an estimator of the memory parameter based on the regression in a neighbourhood of the zero‐frequency of the logarithm of the averaged periodogram, obtained by computing the empirical mean of the periodogram over adjacent epochs. It is shown that replacing the periodogram by its average has an effect similar to the frequency domain pooling to reduce the variance of the estimate. We also propose a simple procedure to test the stationarity of the memory coefficient. A limited Monte Carlo experiment is presented to support our findings.