Forecasting reservoir monthly runoff via ensemble empirical mode decomposition and extreme learning machine optimized by an improved gravitational search algorithm

Monthly streamflow prediction plays a significant role in reservoir operation and water resource management. Hence, this research tries to develop a hybrid model for accurate monthly streamflow prediction, where the ensemble empirical mode decomposition (EEMD) is firstly used to decompose the original streamflow data into a finite amount of intrinsic mode functions (IMFs) and a residue; and then the extreme learning machine (ELM) is employed to forecast each IMFs and the residue, while an improved gravitational search algorithm (IGSA) based on elitist-guide evolution strategies, selection operator and mutation operator is used to select the parameters of all the ELM models; finally, the summarized predicated results for all the subcomponents are treated as the final forecasting result. The hybrid method is applied to forecast the monthly runoff of Three Gorges in China, while four quantitative indexes are used to test the performances of the developed forecasting models. The results show that EEMD can effectively separate the internal characteristics of the original monthly runoff, and the hybrid model is able to make an obvious improvement over other models in hydrological time series prediction.     

Comparison of model selection for regression

We discuss empirical comparison of analytical methods for model selection. Currently, there is no consensus on the best method for finite-sample estimation problems, even for the simple case of linear estimators. This article presents empirical comparisons between classical statistical methods - Akaike information criterion (AIC) and Bayesian information criterion (BIC) - and the structural risk minimization (SRM) method, based on Vapnik-Chervonenkis (VC) theory, for regression problems. Our study is motivated by empirical comparisons in Hastie, Tibshirani, and Friedman (2001), which claims that the SRM method performs poorly for model selection and suggests that AIC yields superior predictive performance. Hence, we present empirical comparisons for various data sets and different types of estimators (linear, subset selection, and k-nearest neighbor regression). Our results demonstrate the practical advantages of VC-based model selection; it consistently outperforms AIC for all data sets. In our study, SRM and BIC methods show similar predictive performance. This discrepancy (between empirical results obtained using the same data) is caused by methodological drawbacks in Hastie et al. (2001), especially in their loose interpretation and application of SRM method. Hence, we discuss methodological issues important for meaningful comparisons and practical application of SRM method. We also point out the importance of accurate estimation of model complexity (VC-dimension) for empirical comparisons and propose a new practical estimate of model complexity for k-nearest neighbors regression.     

Mixture model selection via hierarchical BIC

The Bayesian information criterion (BIC) is one of the most popular criteria for model selection in finite mixture models. However, it implausibly penalizes the complexity of each component using the whole sample size and completely ignores the clustered structure inherent in the data, resulting in over-penalization. To overcome this problem, a novel criterion called hierarchical BIC (HBIC) is proposed which penalizes the component complexity only using its local sample size and matches the clustered data structure well. Theoretically, HBIC is an approximation of the variational Bayesian (VB) lower bound when sample size is large and the widely used BIC is a less accurate approximation. An empirical study is conducted to verify this theoretical result and a series of experiments is performed on simulated and real data sets to compare HBIC and BIC. The results show that HBIC outperforms BIC substantially and BIC suffers from underestimation.     

基于Vapnik-Chervonenkis泛化界的极限学习机模型复杂性控制

模型复杂性是决定学习机器泛化性能的关键因素,对其进行合理的控制是模型选择的重要原则.极限学习机(extreme learning machine,ELM)作为一种新的机器学习算法,表现出了优越的学习性能.但对于如何在ELM的模型选择过程中合理地度量和控制其模型复杂性这一基本问题,目前尚欠缺系统的研究.本文讨论了基于Vapnik-Chervonenkis(VC)泛化界的ELM模型复杂性控制方法(记作VM),并与其他4种经典模型选择方法进行了系统的比较研究.在人工和实际数据集上的实验表明,与其他4种经典方法相比,VM具有更优的模型选择性能:能选出同时具有最低模型复杂性和最低(或近似最低)实际预测风险的ELM模型.此外,本文也为VC维理论的实际应用价值研究提供了一个新的例证.     

Efficiency of classification methods based on empirical risk minimization

A binary classification problem is reduced to the minimization of convex regularized empirical risk functionals in a reproducing kernel Hilbert space. The solution is searched for in the form of a finite linear combination of kernel support functions (Vapnik’s support vector machines). Risk estimates for a misclassification as a function of the training sample size and other model parameters are obtained.     

E-V-ALSTM模型的股价预测

针对股票价格非平稳、非线性、高复杂和随机波动等特性使其预测难度大的问题,提出一种基于E-V-ALSTM混合深度模型的股票价格预测方法。使用经验模态分解（EMD）对股票价格数据进行第一次分解,得到若干固有模态函数（IMFs）和一个残差（Res）,降低了股票价格数据的非平稳性和非线性;使用样本熵（SampEn）对这些IMFs进行复杂性评估;将复杂度高于一定阈值的IMFs使用变分模态分解（VMD）进行二次分解,以降低股票价格数据的复杂性;通过加入注意力机制的长短期记忆神经网络（LSTM）模型进行预测,捕捉关键时间点特征信息,重新赋予权重,以解决股票价格数据的随机波动性,提升预测方法的精确度。对沪深300指数和德国DAX指数等数据集上的实验结果表明,该模型比其他对比模型能进一步提高股票价格预测的准确性。     

Empirical data modeling in software engineering using radial basisfunctions

Many empirical studies in software engineering involve relationships between various process and product characteristics derived via linear regression analysis. We propose an alternative modeling approach using radial basis functions (RBFs) which provide a flexible way to generalize linear regression function. Further, RBF models possess strong mathematical properties of universal and best approximation. We present an objective modeling methodology for determining model parameters using our recent SG algorithm, followed by a model selection procedure based on generalization ability. Finally, we describe a detailed RBF modeling study for software effort estimation using a well-known NASA dataset     

SFFS-PC-NN optimized by genetic algorithm for dynamic prediction of financial distress with longitudinal data streams

Recently, research of financial distress prediction has become increasingly urgent. However, existing static models for financial distress prediction are not able to adapt to the situation that the sample data flows constantly with the lapse of time. Financial distress prediction with static models does not meet the demand of the dynamic nature of business operations. This article explores the theoretical and empirical research of dynamic modeling on financial distress prediction with longitudinal data streams from the view of individual enterprise. Based on enterprise’s longitudinal data streams, dynamic financial distress prediction model is constructed by integrating financial indicator selection by using sequential floating forward selection method, dynamic evaluation of enterprise’s financial situation by using principal component analysis at each longitudinal time point, and dynamic prediction of financial distress by using back-propagation neural network optimized by genetic algorithm. This model’s ex-ante prediction efficiently combines its ex-post evaluation. In empirical study, three listed companies’ half-year longitudinal data streams are used as the sample set. Results of dynamic financial distress prediction show that the longitudinal and dynamic model of enterprise’s financial distress prediction is more effective and feasible than static model.     

Finite sample properties of linear model identification

In this paper, we consider the finite sample properties of prediction error methods using a quadratic criterion function for system identification. The problem we pose is: How many data points are required to guarantee with high probability that the expected value of the quadratic identification criterion is close to its empirical mean value? The sample sizes are obtained using risk minimization theory which provides uniform probabilistic bounds on the difference between the expected value of the squared prediction error and its empirical mean evaluated on a finite number of data points. The bounds are very general. No assumption is made about the true system belonging to the model class, and the noise sequence is not assumed to be uniformly bounded. Further analysis shows that in order to maintain a given bound on the deviation, the number of data points needed grows no faster than quadratically with the number of parameters for FIR and ARX models     

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.     

基于更新样本智能识别算法的自适应集成建模

选择表征建模对象特性漂移的新样本对软测量模型进行自适应更新,能够降低模型复杂度和运行消耗,提高模型可解释性和预测精度.针对新样本近似线性依靠程度（Approximate linear dependence, ALD）和预测误差（Prediction error, PE）等指标只能片面反映建模对象的漂移程度,领域专家结合具体工业过程需要依据上述指标和自身积累经验进行更新样本的有效识别等问题,本文提出了基于更新样本智能识别算法的自适应集成建模策略.首先,基于历史数据离线建立基于改进随机向量泛函连接网络（Improved random vector functional-link networks, IRVFL）的选择性集成模型;然后,基于集成子模型对新样本进行预测输出后采用在线自适应加权算法（On-line adaptive weighting fusion, OLAWF）对集成子模型权重进行更新,实现在线测量阶段对建模对象特性变化的动态自适应;接着基于领域专家知识构建模糊推理模型对新样本相对ALD（Relative ALD, RALD）值和相对PE（Relative PE, RPE）值进行融合,实现更新样本智能识别,构建新的建模样本库;最后实现集成模型的在线自适应更新.采用合成数据仿真验证了所提算法的合理性和有效性.     

基于可解释集成学习的信贷违约预测

人工智能促进了风控行业的发展,智能风控的核心在于风险控制,信贷违约预测模型是解决这一问题必须倚靠的手段.传统的解决方案是基于人工和广义线性模型建立的,然而现在通过网络完成的交易数据,具有高维性和多重来源等特点,远远超出了现有模型的处理能力,对于传统风控提出了巨大的挑战.因此,本文提出一种基于融合方法的可解释信贷违约预测模型,首先选取LightGBM、DeepFM和CatBoost作为基模型,CatBoost作为次模型,通过模型融合提升预测结果的准确性,然后引入基于局部的、与模型无关的可解释性方法LIME,解释融合模型的预测结果.基于真实数据集的实验结果显示,该模型在信贷违约预测任务上具有较好的精确性和可解释性.     

Holistic environmental soil-landscape modeling of soil organic carbon

In environmental soil-landscape modeling (ESLM), the selection of predictive variables is commonly contingent on the researchers' domain expertise on soil–environment processes. This variable selection strategy may suffer bias or even fail in regions where the process knowledge is insufficient. To overcome this problem, this study demonstrates a holistic ESLM framework which consists of five components: model conceptualization, data compilation, process identification, parsimonious model calibration, and model validation. Based on the STEP-AWBH conceptual model, a comprehensive pool of 210 potential environmental variables that exhaustively cover pedogenic and environmental factors was constructed. This was followed by strategic variable selection and development of parsimonious prediction models using machine learning techniques. The all-relevant variable selection successfully identified the major and minor factors relevant to the SOC variation, showing that the major factors important for explaining SOC variation in Florida were vegetation and soil water gradient. Topography and climate showed moderate effects on SOC variation. Parsimonious SOC models developed using four minimal-optimal variable selection techniques and simulated annealing yielded optimal predictive performance with minimal model complexity. The holistic ESLM framework not only provides a new view of selecting and utilizing variables for predicting soil properties but can also assist in identifying the underlying processes of soil-environment systems of interest. Due to the flexibility of the framework to incorporate various types of variable selection and modeling techniques, the holistic environmental modeling strategy can be generalized to other environmental modeling domains for both prediction and process identification.     

Performance evaluation of weights selection schemes for linear combination of multiple forecasts

Time series modeling and forecasting are essential in many domains of science and engineering. Extensive works in literature suggest that combining outputs of different forecasting methods substantially increases the overall accuracies as well as reduces the risk of model selection. The most popular method of forecasts combination is the weighted averaging of the constituent forecasts. The effectiveness of this method solely depends on appropriate selection of the combining weights. In this paper, we comprehensively evaluate a wide variety of benchmark weights selection techniques for linear combination of multiple forecasts in terms of their prediction accuracies. Nine real-world time series from different domains and five individual forecasting methods are used in our empirical work. A robust scheme is also suggested for fairly ranking the combination methods on the basis of their forecasting performances. Our study precisely demonstrates the relative strengths and weaknesses of various benchmark linear combination techniques which evidently can be of much practical importance.     

Nonparametric Time Series Prediction Through Adaptive Model Selection

We consider the problem of one-step ahead prediction for time series generated by an underlying stationary stochastic process obeying the condition of absolute regularity, describing the mixing nature of process. We make use of recent results from the theory of empirical processes, and adapt the uniform convergence framework of Vapnik and Chervonenkis to the problem of time series prediction, obtaining finite sample bounds. Furthermore, by allowing both the model complexity and memory size to be adaptively determined by the data, we derive nonparametric rates of convergence through an extension of the method of structural risk minimization suggested by Vapnik. All our results are derived for general L error measures, and apply to both exponentially and algebraically mixing processes.     

Portfolio selection with the effect of systematic risk diversification: formulation and accelerated gradient algorithm

In this paper, we propose an optimal trade-off model for portfolio selection with the effect of systematic risk diversification, measured by the maximum marginal systematic risk of all the risk contributors. First, the classical portfolio selection model with constraints on allocation of systematic risk is shown to be equivalent to our trade-off model under certain conditions. Then, we transform the trade-off model into a special non-convex and non-smooth composite problem equivalently. Thus a modified accelerated gradient (AG) algorithm can be introduced to solve the composite problem. The efficiency of the algorithm for solving the composite problem is demonstrated by theoretical results on both the convergence rate and the iteration complexity bound. Finally, empirical analysis demonstrates that the proposed model is a preferred tool for active portfolio risk management when compared with the existing models. We also carry out a series of numerical experiments to compare the performance of the modified AG algorithm with the other three first-order algorithms.     

Estimation of the functional dependence of time series for the class of regression functions partially covered by a finite <Emphasis Type="Italic">ε</Emphasis>-net

The problem of estimating the functional dependence of time series on the time index is considered in the case of short data sample. The uniform convergence of an empirical risk functional to a theoretical one is proved for the case where the parametric class of regression functions can be partially covered by a finite ε-net for some of its parameters. The functional dependence for the polynomial class of functions is estimated as an example.     

基于自适应时序分解的空气污染物浓度预测

为准确、有效地预测空气污染物浓度,建立了基于自适应完整集成经验模态分解(CEEMDAN)和排列熵(PE)的门控循环单元(GRU)空气污染物预测模型。首先利用CEEMDAN算法对非线性信号的自适应分解能力将原始序列分解为一组不同频率、复杂度的固有模态函数(IMF)和一个残差分量(REC),其次根据PE算法将复杂度相近的IMF分量和REC一起进行重新组合,最后将重组后的子序列分别使用GRU模型进行预测,并将子序列预测结果相加得到最终预测结果。实验结果表明,基于CEEMDAN-PE-GRU模型预测的误差明显低于其他模型,验证了该模型对空气污染物浓度预测的有效性。     

Sparse probit linear mixed model

Linear mixed models (LMMs) are important tools in statistical genetics. When used for feature selection, they allow to find a sparse set of genetic traits that best predict a continuous phenotype of interest, while simultaneously correcting for various confounding factors such as age, ethnicity and population structure. Formulated as models for linear regression, LMMs have been restricted to continuous phenotypes. We introduce the sparse probit linear mixed model (Probit-LMM), where we generalize the LMM modeling paradigm to binary phenotypes. As a technical challenge, the model no longer possesses a closed-form likelihood function. In this paper, we present a scalable approximate inference algorithm that lets us fit the model to high-dimensional data sets. We show on three real-world examples from different domains that in the setup of binary labels, our algorithm leads to better prediction accuracies and also selects features which show less correlation with the confounding factors.