A holistic multi-objective optimization design procedure for ensemble member generation and selection

In the last few years, machine learning techniques have been successfully applied to solve engineering problems. However, owing to certain complexities found in real-world problems, such as class imbalance, classical learning algorithms may not reach a prescribed performance. There can be situations where a good result on different conflicting objectives is desirable, such as true positive and true negative ratios, or it is important to balance model’s complexity and prediction score. To solve such issues, the application of multi-objective optimization design procedures can be used to analyze various trade-offs and build more robust machine learning models. Thus, the creation of ensembles of predictive models using such procedures is addressed in this work. First, a set of diverse predictive models is built by employing a multi-objective evolutionary algorithm. Next, a second multi-objective optimization step selects the previous models as ensemble members, resulting on several non-dominated solutions. A final multi-criteria decision making stage is applied to rank and visualize the resulting ensembles. To analyze the proposed methodology, two different experiments are conducted for binary classification. The first case study is a famous classification problem through which the proposed procedure is illustrated. The second one is a challenging real-world problem related to water quality monitoring, where the proposed procedure is compared to four classical ensemble learning algorithms. Results on this second experiment show that the proposed technique is able to create robust ensembles that can outperform other ensemble methods. Overall, the authors conclude that the proposed methodology for ensemble generation creates competitive models for real-world engineering problems.     

Dynamic multi-objective optimization based on membrane computing for control of time-varying unstable plants

Dynamic multi-objective optimization is a current hot topic. This paper discusses several issues that has not been reported in the static multi-objective optimization literature such as the loss of non-dominated solutions, the emergence of the false non-dominated solutions and the necessity for an online decision-making mechanism. Then, a dynamic multi-objective optimization algorithm is developed, which is inspired by membrane computing. A novel membrane control strategy is proposed in this article and is applied to the optimal control of a time-varying unstable plant. Experimental results clearly illustrate that the control strategy based on the dynamic multi-objective optimization algorithm is highly effective with a short rise time and a small overshoot.     

异构集成代理辅助多目标粒子群优化算法

针对代理辅助进化算法在减少昂贵适应度评估时难以通过少量样本点构造高质量代理模型的问题,提出异构集成代理辅助多目标粒子群优化算法。该方法通过使用加权平均法将Kriging模型和径向基函数网络模型组合成高精度的异构集成模型,达到增强算法处理不确定性信息能力的目的。基于集成学习的两种代理模型分别应用于全局搜索和局部搜索,在多目标粒子群优化算法框架基础上,新提出的方法为每个目标函数自适应地构造了异构集成模型,利用其模型的非支配解来指导粒子群的更新,得出目标函数的最优解集。实验结果表明,所提方法提高了代理模型的搜索能力,减少了评估次数,并且随着搜索维度的增加,其计算复杂性也具有更好的可扩展性。     

基于深度学习特征提取与多目标优化集成修剪的选择性集成学习软测量方法

集成学习已成为一种广泛使用的软测量建模框架,但是建立高性能的集成学习软测量模型依然面临特征选择不当、基模型多样性不足、基模型估计性能不佳等诸多挑战.为此,提出一种基于堆栈自编码器多样性生成机制的选择性集成学习高斯过程回归(selective ensemble of stacked autoencoder based Gaussian process regression, SESAEGPR)软测量建模方法.该方法充分发挥深度学习在特征提取方面的优势,通过构建多样性的堆栈自编码器(stacked autoencoder, SAE)网络,建立基于隐特征的高斯过程回归(Gaussian process regression, GPR)基模型.基于模型性能提升率和进化多目标优化对SAEGPR基模型实施两次集成修剪,以降低集成模型复杂度、保持甚至进一步提升模型估计性能,最后,引入PLS Stacking集成策略实现基模型融合.所提出方法显著优于传统全局和全集成软测量建模方法,其有效性和优越性通过青霉素发酵过程和Tennessee Eastman化工过程得到验证.     

Variable weights combined model based on multi-objective optimization for short-term wind speed forecasting

Accurate and steady wind speed prediction is essential for the efficient management of wind power factories and energy systems. However, it is difficult to obtain satisfactory forecasting performance because of the characteristics of random nonlinear fluctuations inherent in wind speed variation. Considering the drawbacks of statistical models in forecasting nonlinear time series and the problem of artificial intelligence models easily falling into a local optimum, in this study, we successfully integrate the variable weighted combination theory into a new combined forecasting model that simultaneously consists of three disparate hybrid models based on the decomposition technology. Moreover, the extreme learning machine optimized by the multi-objective grasshopper optimization algorithm is adopted to integrate all the forecasting results derived from each hybrid model to further enhance the forecasting accuracy. In this study, we consider a case study that employs several authentic wind speed data aggregates of Shandong wind farms for an evaluation of the forecasting performance of the proposed combined model. The experimental results reveal that this proposed model surpasses the contrasted benchmark models and is satisfactory for intellective grid programs.     

Wind Speed Prediction Using Chicken Swarm Optimization with Deep Learning Model

High precision and reliable wind speed forecasting have become a challenge for meteorologists. Convective events, namely, strong winds, thunderstorms, and tornadoes, along with large hail, are natural calamities that disturb daily life. For accurate prediction of wind speed and overcoming its uncertainty of change, several prediction approaches have been presented over the last few decades. As wind speed series have higher volatility and nonlinearity, it is urgent to present cutting-edge artificial intelligence (AI) technology. In this aspect, this paper presents an intelligent wind speed prediction using chicken swarm optimization with the hybrid deep learning (IWSP-CSODL) method. The presented IWSP-CSODL model estimates the wind speed using a hybrid deep learning and hyperparameter optimizer. In the presented IWSP-CSODL model, the prediction process is performed via a convolutional neural network (CNN) based long short-term memory with autoencoder (CBLSTMAE) model. To optimally modify the hyperparameters related to the CBLSTMAE model, the chicken swarm optimization (CSO) algorithm is utilized and thereby reduces the mean square error (MSE). The experimental validation of the IWSP-CSODL model is tested using wind series data under three distinct scenarios. The comparative study pointed out the better outcomes of the IWSP-CSODL model over other recent wind speed prediction models.     

基于隐性记忆的非平稳时变污水处理过程多目标运行优化

针对污水处理过程中运行能耗大和水质超标严重等问题,基于隐性记忆策略,提出一种适用于无规律且非平稳时变过程的多目标运行优化方法.首先,采用集成即时学习在线建模算法建立运行指标模型,实现时变工况下运行优化目标的准确描述;然后,提出基于隐性记忆的多目标优化算法,通过充分考虑复杂时变工况,挖掘埋藏在历史优化信息中的结构化知识,引导进化搜索过程,并结合平均距离聚类指标进行均匀随机局部搜索,提高算法收敛性、多样性和分布性;最后,利用基于集成即时学习的智能决策方法,获取操作变量溶解氧浓度和硝态氮浓度的优化设定值并进行跟踪控制.数据实验表明:所提出方法能够实现污水处理过程的稳定运行,在满足操作限制的情况下,提高污染物去除性能,降低运行成本.     

一种新的基于多样性赋权证据推理的集成学习方法

在集成学习中使用平均法、投票法作为结合策略无法充分利用基分类器的有效信息,且根据波动性设置基分类器的权重不精确、不恰当。以上问题会降低集成学习的效果,为了进一步提高集成学习的性能,提出将证据推理(evidence reasoning, ER)规则作为结合策略,并使用多样性赋权法设置基分类器的权重。首先,由多个深度学习模型作为基分类器、ER规则作为结合策略,构建集成学习的基本结构;然后,通过多样性度量方法计算每个基分类器相对于其他基分类器的差异性;最后,将差异性归一化实现基分类器的权重设置。通过多个图像数据集的分类实验,结果表明提出的方法较实验选取的其他方法准确率更高且更稳定,证明了该方法可以充分利用基分类器的有效信息,且多样性赋权法更精确。     

Ensemble learning for wind profile prediction with missing values

In this paper, we aim to develop computational intelligence approaches for wind profile prediction. Specifically, we focus on two aspects in this work. First, we investigate the missing value recovery for wind data. Due to the complexity of data collection in such processes, wind data normally include missing values. Therefore, how to effectively recover such missing values for learning and prediction is an important aspect for wind profile prediction. Second, we develop an ensemble learning approach based on multiple neural network models. Our proposed method uses a new strategy based on the temporal information to assign the weights for each model dedicated for wind profile prediction to achieve better prediction performance. Various simulation studies and statistical testing demonstrate the effectiveness of our approach.     

Wind power prediction using deep neural network based meta regression and transfer learning

An innovative short term wind power prediction system is proposed which exploits the learning ability of deep neural network based ensemble technique and the concept of transfer learning. In the proposed DNN-MRT scheme, deep auto-encoders act as base-regressors, whereas Deep Belief Network is used as a meta-regressor. Employing the concept of ensemble learning facilitates robust and collective decision on test data, whereas deep base and meta-regressors ultimately enhance the performance of the proposed DNN-MRT approach. The concept of transfer learning not only saves time required during training of a base-regressor on each individual wind farm dataset from scratch but also stipulates good weight initialization points for each of the wind farm for training. The effectiveness of the proposed, DNN-MRT technique is expressed by comparing statistical performance measures in terms of root mean squared error (RMSE), mean absolute error (MAE), and standard deviation error (SDE) with other existing techniques.     

基于混合预测策略与改进社会学习优化算法的动态多目标优化方法

近年来,国内外学者针对基于预测的动态多目标优化算法开展了深入研究,并提出了一系列有效的算法,然而已有的研究工作通常采用单一的预测策略,使得算法不能有效地应对剧烈的环境变化。针对上述问题,提出了一种基于混合预测策略与改进社会学习优化算法的动态多目标优化方法。具体地,当环境发生变化时,该方法首先基于代表性个体预测策略生成一部分群体;其次,基于拐点预测策略生成一部分新群体,特别地,为了提高种群的多样性,根据算法迭代的历史信息和环境变化情况随机地生成一定数量的新个体;为了提高问题的求解效率,对社会学习优化算法进行了改进,为每个进化空间设计了适用于动态多目标优化问题的算子;最后,将混合预测策略与改进的社会学习优化算法结合,构成了一种新的动态多目标优化方法。以FDA、DMOP和F函数集作为实验测试函数集,与四种代表性算法进行了性能对比;并以反向世代距离(inverted generational distance, IGD)对该方法的性能进行了深入的分析。实验结果表明所提方法具有较好的收敛性和鲁棒性。     

基于分解的预测型动态多目标粒子群优化算法

针对动态多目标问题求解,提出一种基于分解的预测型动态多目标粒子群优化算法.首先借助分解思想,将目标问题划分为多个不同的子问题,当问题动态变化时,选择对应于不同子问题的优化个体检测环境变化程度,以提高算法对不同动态问题的适应与响应能力;然后,设计一种群体预测策略,通过将目标空间中相同收敛方向上不同时刻的个体位置转换为时间序列,引入时间序列预测方法预测下一刻位置,从而提高预测种群的多样性和有效性,进而有效减少算法在问题变化后的收敛时间;最后,为避免问题发生变化后个体与子问题不匹配,设计一种再匹配策略,以提高预测策略的准确性.实验结果表明,在6个标准动态多目标测试问题上,与2个动态多目标优化算法进行比较,所提出算法在收敛性、分布性与稳定性上均具有显著优势.     

A SVR-based ensemble approach for drifting data streams with recurring patterns

Pattern drift is a common issue for machine learning in real applications, as the distribution generating the data may change under nonstationary environmental/operational conditions. In our previous work, a strategy based on Feature Vector Selection (FVS) has been proposed for enabling a Support Vector Regression (SVR) model to adaptively update with streaming data, but the proposed strategy suffers from the incapability of treating recurring patterns. An instance-based online learning approach is proposed in this paper, which can adaptively update an SVR-based ensemble model with steaming data points. The proposed approach reduces the computational complexity of the updating process by selecting only part of the newly available data and allows following timely the ongoing patterns by resorting to FVS. The proposed approach creates new sub-models directly from a basic model and the sub-models represent separately the data stream at different periods. A dynamic ensemble selection strategy is integrated in the approach to select the sub-models most relevant to the new data point for deriving the prediction, while reducing the influence of the irrelevant ones. The weights of the different models in the ensemble are updated, based on their prediction errors. Comparison results with several benchmark approaches on several synthetic datasets and on the dataset concerning the leakage from the first seal in a Reactor Coolant Pump, prove the efficiency and accuracy of the proposed online learning ensemble approach.     

Stacking Deep learning and Machine learning models for short-term energy consumption forecasting

Accurate prediction of electricity consumption is essential for providing actionable insights to decision-makers for managing volume and potential trends in future energy consumption for efficient resource management. A single model might not be sufficient to solve the challenges that result from linear and non-linear problems that occur in electricity consumption prediction. Moreover, these models cannot be applied in practice because they are either not interpretable or poorly generalized. In this paper, a stacking ensemble model for short-term electricity consumption is proposed. We experimented with machine learning and deep models like Random Forests, Long Short Term Memory, Deep Neural Networks, and Evolutionary Trees as our base models. Based on the experimental observations, two different ensemble models are proposed, where the predictions of the base models are combined using Gradient Boosting and Extreme Gradient Boosting (XGB). The proposed ensemble models were tested on a standard dataset that contains around 500,000 electricity consumption values, measured at periodic intervals, over the span of 9 years. Experimental validation revealed that the proposed ensemble model built on XGB reduces the training time of the second layer of the ensemble by a factor of close to 10 compared to the state-of-the-art , and also is more accurate. An average reduction of approximately 39% was observed in the Root mean square error.     

基于参考输入优化的变速风电机组最大化风能捕获方法

变速风电机组在额定风速以下应用最大功率点跟踪实现最大化风能捕获. 然而, 大惯量风电机组在面对快 速波动的湍流风速时, 因转速调节慢而难以保持运行于最大功率点. 本文研究进一步发现, 平均转速跟踪误差与整 体的风能捕获效率并非单调关系, 这使得当前以减小转速跟踪误差为目标的控制器设计难以有效提升风电机组的 发电效率. 为此, 本文以提升风能捕获效率(而非减小转速跟踪误差)为目标, 提出一种基于参考输入优化的风电机 组最大化风能捕获方法. 考虑到参考转速对风能捕获效率的复杂影响难以准确建模, 本文借助深度确定性策略梯度 (DDPG)强化学习算法实现参考输入优化. 仿真结果表明该方法能够有效提升湍流风下变速风电机组的风能捕获效 率.     

考虑无线充电的无人机路径在线规划

近年来, 无人机在物流、通信、军事任务、灾害救援等领域中展现出了巨大的应用潜力, 然而无人机的续航 能力是制约其使用的重大因素, 在无线充电技术不断突破和发展的背景下, 本文基于深度强化学习方法, 提出了一 种考虑无线充电的无人机路径在线优化方法, 通过无线充电技术提高无人机的任务能力. 首先, 对无人机功耗模型 和无线充电模型进行了构建, 根据无人机的荷电状态约束, 设计了一种基于动态上下文向量的深度神经网络模型, 通过编码器和解码器的模型架构, 实现无人机路径的直接构造, 通过深度强化学习方法对模型进行离线训练, 从而 应用于考虑无线充电的无人机任务路径在线优化. 文本通过与传统优化方法和深度强化学习方法进行实验对比, 所提方法在CPU算力和GPU算力下分别实现了4倍以及100倍以上求解速度的提升.     

基于异构分类器集成的增量学习算法

将集成学习的思想引入到增量学习之中可以显著提升学习效果,近年关于集成式增量学习的研究大多采用加权投票的方式将多个同质分类器进行结合,并没有很好地解决增量学习中的稳定-可塑性难题。针对此提出了一种异构分类器集成增量学习算法。该算法在训练过程中,为使模型更具稳定性,用新数据训练多个基分类器加入到异构的集成模型之中,同时采用局部敏感哈希表保存数据梗概以备待测样本近邻的查找;为了适应不断变化的数据,还会用新获得的数据更新集成模型中基分类器的投票权重;对待测样本进行类别预测时,以局部敏感哈希表中与待测样本相似的数据作为桥梁,计算基分类器针对该待测样本的动态权重,结合多个基分类器的投票权重和动态权重判定待测样本所属类别。通过对比实验,证明了该增量算法有比较高的稳定性和泛化能力。     

基于参考线的预测策略求解动态多目标优化问题

为了快速且准确地跟踪动态多目标优化问题变化的Pareto前沿与Pareto解集,在可以不依靠历史信息的前提下,提出一种基于参考线预测策略的求解动态多目标优化问题的算法(RLPS).该算法通过记录每个参考线关联的种群个体在环境变化初始时和个体自主进化一小段时间后个体位置的变化,预测最优个体所在方向,同时在该方向上均匀分布若干延伸个体,选出每个参考线关联的非支配个体作为当前环境下的引导个体,在选出的引导个体邻域内随机产生若干伴随个体增加种群多样性.通过5个标准动态测试函数对该算法测试,并与两个现有算法作对比分析,实验结果表明所提出的算法具有更快地响应环境变化的能力.     

Artificial immune system in dynamic environments solving time-varying non-linear constrained multi-objective problems

A bio-inspired artificial immune system is developed to track dynamically the Pareto fronts of time-varying constrained multi-objective problems with changing variable dimensions. It executes in order T-module, B-module, and M-module within a run period. The first module is designed to examine dynamically whether the environment changes or whether a change takes place in the optimization problem, while creating an initial population by means of the history information. Thereafter, the second one is a loop of optimization that searches for the desired non-dominated front of a given environment, in which the evolving population is sorted into several subpopulations. Each of such subpopulations, relying upon the population diversity, suppresses its redundant individuals and evolves the winners. The last one stores temporarily the resultant non-dominated solutions of the environment that assist T-module to create some initial candidates helpful for the coming environment. These dynamic characteristics, along with the comparative experiments guarantee that the artificial immune system can track adaptively the time-varying environment and maintain the diversity of population while being of potential use for complex dynamic constrained multi-objective problems.     

Probabilistic wind speed forecasting using Bayesian model averaging with truncated normal components

Bayesian model averaging (BMA) is a statistical method for post-processing forecast ensembles of atmospheric variables, obtained from multiple runs of numerical weather prediction models, in order to create calibrated predictive probability density functions (PDFs). The BMA predictive PDF of the future weather quantity is the mixture of the individual PDFs corresponding to the ensemble members and the weights and model parameters are estimated using forecast ensembles and validating observations from a given training period. A BMA model for calibrating wind speed forecasts is introduced using truncated normal distributions as conditional PDFs and the method is applied to the ALADIN-HUNEPS ensemble of the Hungarian Meteorological Service and to the University of Washington Mesoscale Ensemble. Three parameter estimation methods are proposed and each of the corresponding models outperforms the traditional gamma BMA model both in calibration and in accuracy of predictions.