Stacking Deep learning and Machine learning models for short-term energy consumption forecasting

Accurate prediction of electricity consumption is essential for providing actionable insights to decision-makers for managing volume and potential trends in future energy consumption for efficient resource management. A single model might not be sufficient to solve the challenges that result from linear and non-linear problems that occur in electricity consumption prediction. Moreover, these models cannot be applied in practice because they are either not interpretable or poorly generalized. In this paper, a stacking ensemble model for short-term electricity consumption is proposed. We experimented with machine learning and deep models like Random Forests, Long Short Term Memory, Deep Neural Networks, and Evolutionary Trees as our base models. Based on the experimental observations, two different ensemble models are proposed, where the predictions of the base models are combined using Gradient Boosting and Extreme Gradient Boosting (XGB). The proposed ensemble models were tested on a standard dataset that contains around 500,000 electricity consumption values, measured at periodic intervals, over the span of 9 years. Experimental validation revealed that the proposed ensemble model built on XGB reduces the training time of the second layer of the ensemble by a factor of close to 10 compared to the state-of-the-art , and also is more accurate. An average reduction of approximately 39% was observed in the Root mean square error.     

Adaptive water demand forecasting for near real-time management of smart water distribution systems

This paper presents a novel methodology to perform adaptive Water Demand Forecasting (WDF) for up to 24 h ahead with the aim to support near real-time operational management of smart Water Distribution Systems (WDSs). The novel WDF methodology is exclusively based on the analysis of water demand time series (i.e., demand signals) and makes use of Evolutionary Artificial Neural Networks (EANNs). It is implemented in a fully automated, data-driven and self-learning Demand Forecasting System (DFS) that is readily transferable to practice. The main characteristics of the DFS are: (a) continuous adaptability to ever changing water demand patterns and (b) generic and seamless applicability to different demand signals. The DFS enables applying two alternative WDF approaches. In the first approach, multiple EANN models are used in parallel to separately forecast demands for different hours of the day. In the second approach, a single EANN model with a fixed forecast horizon (i.e., 1 h) is used in a recursive fashion to forecast demands. Both approaches have been tested and verified on a real-life WDS in the United Kingdom (UK). The results obtained illustrate that, regardless of the WDF approach used, the novel methodology allows accurate forecasts to be generated thereby demonstrating the potential to yield substantial improvements to the state-of-the-art in near real-time WDS management. The results obtained also demonstrate that the multiple-EANN-models approach slightly outperforms the single-EANN-model approach in terms of WDF accuracy. The single-EANN-model approach, however, still enables achieving good WDF performance and may be a preferred option in engineering practice as it is easier to setup/implement.     

Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting

Load demand forecasting is a critical process in the planning of electric utilities. An ensemble method composed of Empirical Mode Decomposition (EMD) algorithm and deep learning approach is presented in this work. For this purpose, the load demand series were first decomposed into several intrinsic mode functions (IMFs). Then a Deep Belief Network (DBN) including two restricted Boltzmann machines (RBMs) was used to model each of the extracted IMFs, so that the tendencies of these IMFs can be accurately predicted. Finally, the prediction results of all IMFs can be combined by either unbiased or weighted summation to obtain an aggregated output for load demand. The electricity load demand data sets from Australian Energy Market Operator (AEMO) are used to test the effectiveness of the proposed EMD-based DBN approach. Simulation results demonstrated attractiveness of the proposed method compared with nine forecasting methods.     

考虑舒适度的制造环境中HVAC节能优化控制研究

针对制造系统中HVAC高能耗的问题,通过分析制造环境中的热量对HVAC与温度的影响,建立了考虑舒适度的HVAC节能优化模型,并运用模拟退火算法优化目标函数.实验结果表明,HVAC节能优化模型不仅降低了5.9％的能耗,而且室内温度范围在28～29℃,符合节能与舒适的双标准.     

Selective presentation learning for neural network forecasting of stock markets

This paper proposes a selective presentation learning technique for improving the learnability and predictability of large changes by back-propagation neural networks. Daily stock prices are predicted as a complicated real-world problem, taking non-numerical factors such as political and international events into account. Training data corresponding to large changes of prediction-target time series are presented more often, and network learning is stopped at the point that has the maximal profit. When this technique is applied to daily stock-price prediction, the prediction error on large-change data was reduced by 11%, and the network's ability to make profits through experimental stock-trading was improved by 67% to 81%, in comparison with results obtained using conventional learning techniques.     

Deep convolutional learning for general early design stage prediction models

Designers rely on performance predictions to direct the design toward appropriate requirements. Machine learning (ML) models exhibit the potential for rapid and accurate predictions. Developing conventional ML models that can be generalized well in unseen design cases requires an effective feature engineering and selection. Identifying generalizable features calls for good domain knowledge by the ML model developer. Therefore, developing ML models for all design performance parameters with conventional ML will be a time-consuming and expensive process. Automation in terms of feature engineering and selection will accelerate the use of ML models in design.Deep learning models extract features from data, which aid in model generalization. In this study, we (1) evaluate the deep learning model’s capability to predict the heating and cooling demand on unseen design cases and (2) obtain an understanding of extracted features. Results indicate that deep learning model generalization is similar to or better than that of a simple neural network with appropriate features. The reason for the satisfactory generalization using the deep learning model is its ability to identify similar design options within the data distribution. The results also indicate that deep learning models can filter out irrelevant features, reducing the need for feature selection.     

Comprehensive preference learning and feature validity for designing energy-efficient residential buildings using machine learning paradigms

Having a reliable approximation of heating load (HL) and cooling load (CL) is a substantial task for evaluating the energy performance of buildings (EPB). Also, the appearance of soft computing techniques has made many traditional methods antiquated. Thus, the main effort of this study was to evaluate the capability of several learning methods for appraising the HL and CL of a residential building. To this end, a proper dataset consisting of eight influential factors was provided. To simplify the problem, we executed feature validity by using a correlation-based feature subset selection (CfsSubsetEval) technique. The results of this process showed that wall area, overall height, orientation and glazing area have the most significant impact on the HL and CL simulation. After preparing the suitable dataset, sixteen learning methods namely, elastic net (EN), Gaussian process regression (GPR), least median of squares regression (LMSR), multiple linear regression (MLR), multi-layer perceptron regression (MPR), multi-layer perceptron (MLP), radial basis function regression (RBFR), sequential minimal optimization regression (SMOR), functions XNV, lazy K-star, lazy LWL, rules decision table (RDT), M5Rules, alternating model tree (AMT), directional path consistency (DPC), and Random Forest (RF) were developed in Weka environment to forecast the HL and CL variables. Referring to the results, it was concluded that RF, lazy K-star, RDT and AMT outperform other predictive models. Also, comparing the results with the results of the previous studies showed that the applied feature reduction not only did not disturb the learning process but also has enhanced the performance of models. Also, due to the excellent accuracy of the MLP, a formula was derived from the optimized structure of it to predict the HL and CL variables.     

Data science and reinforcement learning for price forecasting and raw material procurement in petrochemical industry

Petrochemical industry is one of the major sectors contributing to the world-wide economy and the digital transformation is urgent to enhance core competence. In general, ethylene, propylene and butadiene, which are associated with synthetic chemicals, are the main raw materials of this industry with around 70–80% cost structure. In particular, butadiene is one of the key materials for producing synthetic rubber and used for several daily commodities. However, the price of butadiene fluctuates along with the demand–supply mismatch or by the international economy and political events. This study proposes two-stage data science framework to predict the weekly price of butadiene and optimize the procurement decision. The first stage suggests several the price prediction models with a comprehensive information including contract price, supply rate, demand rate, and upstream and downstream information. The second stage applies the analytic hierarchy process and reinforcement learning technique to derive an optimal policy of procurement decision and reduce the total procurement cost. An empirical study is conducted to validate the proposed framework, and the results improve the accuracy of price forecasts and the procurement cost reduction of the raw materials.     

Deep learning side-channel attack against hardware implementations of AES

In the field of image recognition, machine learning technologies, especially deep learning, have been rapidly advancing alongside the advances of hardware such as GPUs. In image recognition, in general, large numbers of labeled images to be identified are input to a neural network, and repeatedly learning the images enables the neural network to identify objects with high accuracy. A new profiling side-channel attack method, the deep learning side-channel attack (DL-SCA), utilizes the neural network’s high identifying ability to unveil a cryptographic module’s secret key from side-channel information. In DL-SCAs, the neural network is trained with power waveforms captured from a target cryptographic module, and the trained network extracts the leaky part that depends on the secret. However, at this stage, the main target of investigation has been software implementation, and studies regarding hardware implementation, such as ASIC, are somewhat lacking. In this paper, we first depict deep learning techniques, profiling side-channel attacks, and leak models to clarify the relation between secret and side channels. Next, we investigate the use of DL-SCA against hardware implementations of AES and discuss the problem derived from the Hamming distance model and ShiftRow operation of AES. To solve the problem, we propose a new network training method called “mixed model dataset based on round-round XORed value.” We prove that our proposal solves the problem and gives the attack capability to neural networks. We also compare the attack performance and characteristics of DL-SCA to conventional analysis methods such as correlation power analysis and conventional template attack. In our experiment, a dedicated ASIC chip for side-channel analysis is utilized and the chip is also equipped with a side-channel countermeasure AES. We show how DL-SCA can recover secret keys against the side-channel countermeasure circuit. Our results demonstrate that DL-SCA can be a more powerful option against side-channel countermeasure implementations than conventional SCAs.     

Deep learning based online metallic surface defect detection method for wire and arc additive manufacturing

Wire and arc additive manufacturing (WAAM) is an emerging manufacturing technology that is widely used in different manufacturing industries. To achieve fully automated production, WAAM requires a dependable, efficient, and automatic defect detection system. Although machine learning is dominant in the object detection domain, classic algorithms have defect detection difficulty in WAAM due to complex defect types and noisy detection environments. This paper presents a deep learning-based novel automatic defect detection solution, you only look once (YOLO)-attention, based on YOLOv4, which achieves both fast and accurate defect detection for WAAM. YOLO-attention makes improvements on three existing object detection models: the channel-wise attention mechanism, multiple spatial pyramid pooling, and exponential moving average. The evaluation on the WAAM defect dataset shows that our model obtains a 94.5 mean average precision (mAP) with at least 42 frames per second. This method has been applied to additive manufacturing of single-pass, multi-pass deposition and parts. It demonstrates its feasibility in practical industrial applications and has potential as a vision-based methodology that can be implemented in real-time defect detection systems.     

Deep reinforcement learning for a color-batching resequencing problem

In automotive paint shops, changes of colors between consecutive production orders cause costs for cleaning the painting robots. It is a significant task to re-sequence orders and group orders with identical color as a color batch to minimize the color changeover costs. In this paper, a Color-batching Resequencing Problem (CRP) with mix bank buffer systems is considered. We propose a Color-Histogram (CH) model to describe the CRP as a Markov decision process and a Deep Q-Network (DQN) algorithm to solve the CRP integrated with the virtual car resequencing technique. The CH model significantly reduces the number of possible actions of the DQN agent, so that the DQN algorithm can be applied to the CRP at a practical scale. A DQN agent is trained in a deep reinforcement learning environment to minimize the costs of color changeovers for the CRP. Two experiments with different assumptions on the order attribute distributions and cost metrics were conducted and evaluated. Experimental results show that the proposed approach outperformed conventional algorithms under both conditions. The proposed agent can run in real time on a regular personal computer with a GPU. Hence, the proposed approach can be readily applied in the production control of automotive paint shops to resolve order-resequencing problems.     

Deep learning based robot cognitive architecture for collaborative assembly tasks

As the manufacturing industry becomes more agile, the use of collaborative robots capable of safely working with humans is becoming more prevalent, while adaptable and natural interaction is a goal yet to be achieved. This work presents a cognitive architecture composed of perception and reasoning modules that allows a robot to adapt its actions while collaborating with humans in an assembly task. Human action recognition perception is performed using convolutional neural network models with inertial measurement unit and skeleton tracking data. The action predictions are used for task status reasoning which predicts the time left for each action in a task allowing a robot to plan future actions. The task status reasoning uses a recurrent neural network method which is developed for transferability to new actions and tasks. Updateable input parameters allowing the system to optimise for each user and task with each trial performed are also investigated. Finally, the complete system is demonstrated with the collaborative assembly of a small chair and wooden box, along with a solo robot task of stacking objects performed when it would otherwise be idle. The human actions recognised are using a screw driver, Allen key, hammer and hand screwing, with online accuracies between 83–92%. User trials demonstrate the robot deciding when to start collaborative actions in order to synchronise with the user, as well as deciding when it has time to complete an action on its solo task before a collaborative action is required.     

基于二次分解重构策略的航空客流需求预测

考虑到航空客流需求序列的季节性、非线性和非平稳等特点,提出了一个基于二次分解重构策略的航空客流需求预测模型。首先,通过STL和自适应噪声互补集成经验模态分解（CEEMDAN）方法对航空客流需求序列进行二次分解,并根据数据复杂度和相关度的特征分析结果进行分量重构;然后,采用模型匹配策略分别选取自回归单整移动平均季节（SARIMA）、自回归单整移动平均（ARIMA）、核极限学习机（KELM）和双向长短期记忆（BiLSTM）网络模型对各重构分量进行预测,其中KELM和BiLSTM模型的超参数通过自适应树Parzen估计（ATPE）算法确定;最后,将重构分量预测结果进行线性集成。以北京首都国际机场、深圳宝安国际机场和海口美兰国际机场的航空客流数据作为研究对象进行了1步和多步预测实验,实验结果表明,与一次分解集成模型STL-SAAB相比,所提模型的均方根误差（RMSE）提升了14.98%～60.72%。可见以“分而治之”思想为指导,所提模型结合模型匹配和重构策略挖掘出了数据的内在发展规律,从而为科学预判航空客流需求变化趋势提供了新思路。     

Deep learning for depression recognition with audiovisual cues: A review

With the acceleration of the pace of work and life, people are facing more and more pressure, which increases the probability of suffering from depression. However, many patients may fail to get a timely diagnosis due to the serious imbalance in the doctor–patient ratio in the world. A promising development is that physiological and psychological studies have found some differences in speech and facial expression between patients with depression and healthy individuals. Consequently, to improve current medical care, Deep Learning (DL) has been used to extract a representation of depression cues from audio and video for automatic depression detection. To classify and summarize such research, we introduce the databases and describe objective markers for automatic depression estimation. We also review the DL methods for automatic detection of depression to extract a representation of depression from audio and video. Lastly, we discuss challenges and promising directions related to the automatic diagnoses of depression using DL.     

Deep learning techniques for recommender systems based on collaborative filtering

In the Big Data Era, recommender systems perform a fundamental role in data management and information filtering. In this context, Collaborative Filtering (CF) persists as one of the most prominent strategies to effectively deal with large datasets and is capable of offering users interesting content in a recommendation fashion. Nevertheless, it is well-known CF recommenders suffer from data sparsity, mainly in cold-start scenarios, substantially reducing the quality of recommendations. In the vast literature about the aforementioned topic, there are numerous solutions, in which the state-of-the-art contributions are, in some sense, conditioned or associated with traditional CF methods such as Matrix Factorization (MF), that is, they rely on linear optimization procedures to model users and items into low-dimensional embeddings. To overcome the aforementioned challenges, there has been an increasing number of studies exploring deep learning techniques in the CF context for latent factor modelling. In this research, authors conduct a systematic review focusing on state-of-the-art literature on deep learning techniques applied in collaborative filtering recommendation, and also featuring primary studies related to mitigating the cold start problem. Additionally, authors considered the diverse non-linear modelling strategies to deal with rating data and side information, the combination of deep learning techniques with traditional CF-based linear methods, and an overview of the most used public datasets and evaluation metrics concerning CF scenarios.     

贵州主动配电网冷热电三联供系统运行测试分析

本文针对能源互联网中高比例分布式电源消纳难题开展研究,目的是通过测试系统将示范工程中提高能源综合利用效率更直观的体现出来。在贵州主动配电网示范工程基础上搭建了一种适用于主动配电网的冷热电三联供测试系统,建立了相应的分析模型,且进行了现场测试,并对现场测试数据进行运行效益分析,示范工程冷热电三联供系统在90%额定功率工况下综合能源利用效率可达90%以上,较传统冷热电三联供有较大提高。且该系统能够实现在同样在该热（冷）负荷下,其上网电功率可在一定范围内调节,降低天然气价格和提高上网电价是适用于主动配电网的冷热电三联供保证经济运行的基础,对于用电成本高且有供暖供冷需求的用户,冷热电三联供系统有较好的经济性。     

Convolutional neural network: Deep learning-based classification of building quality problems

The rapid development of the construction industry in China has introduced unprecedented quality-related problems in the country’s building industry. In response to this issue, the government has established various complaint channels to report quality problems. Therefore, building quality complaints (BQCs) need to be classified and solved by respective agencies or departments rapidly for avoiding adverse impact on the safety, health, and well-being of people. However, the current process of classifying BQCs is labor intensive, time consuming, and error prone. An automatic complaint classification is required to improve the effectiveness and efficiency of complaint handling, but studies on this issue are limited. Prevailing text classification research in construction has focused on utilizing conventional shallow machine learning. By contrast, this study explores a novel convolutional neural network (CNN)-based approach that incorporates a deep-learning method to automatically classify the short texts contained within BQCs. The presented approach enables capturing the semantic features in BQC texts and automatic classification of the BQCs into predefined categories. After the model optimization, tests are conducted to examine the practical application of the text classification approach compared with Bayes-based and support vector machine classifiers. Results indicate that the developed CNN-based approach performs well in the Chinese BQC classification with limited manual intervention and few complicated feature engineering.     

Deep learning MRI signature to predict survival and treatment benefit from temozolomide in IDH-wildtype glioblastoma

BackgroundO⁶-methylguanine-DNA-methyltransferase (MGMT) methylation status does not correlate with temozolomide (TMZ) sensitivity in all IDH-wildtype glioblastoma (GBM) patients. New predictors of TMZ benefit are still in demand.MethodsBased on MR images, a deep learning image signature was constructed to predict the survival and benefit of temozolomide in patients with IDH wild-type glioblastoma.ResultsDiS signature was associated with OS as an independent prognostic factor in patients with IDH-wildtype glioblastoma. For high-risk group patients, TMZ was associated with improved OS for patients in MGMT-methylated subgroup (HR: 2.051, 95 % CI: 0.939–4.482, log-rank P = 0.034), but had not effect on MGMT-unmethylated patients. However, patients in the low- risk group did not benefit from TMZ.ConclusionDiS could offer complementary value beyond MGMT methylation status in predicting survival benefit from TMZ chemotherapy in patients with IDH-wildtype glioblastoma.     

小型PLC在核电站核岛通风控制中的应用

为实现对CPR1000核电机组核岛通风系统的可靠控制,采用了小型PLC作为其控制器.介绍了该类型核电机组核岛通风控制系统大分散、小集中的特点,对比了小型PLC和直接数字控制器(DDC)两种控制器的性能参数,详细分析了小型PLC应用的优势.实际应用结果验证了小型PLC应用的可行性,对于同类型的控制方式选择具有参考和借鉴作用.     

Machine learning applications in detecting rip channels from images

Images containing rip channels are used in oceanographic studies and can be preprocessed for these studies by identifying which regions of the image contain rip channels. For thousands of images, this process can become cumbersome. In recent years, object detection has become a successful approach for identifying regions of an image. There are several different algorithms for detecting objects from images, however, there is no guidance as to which algorithm works well for detecting rip channels. This paper aims to compare and explore state-of-the-art machine learning algorithms, including the Viola–Jones algorithm, convolution neural networks, and a meta-learner on a dataset of rip channel images. Along with the comparison, another objective is to find suitable features for rip channels and to implement the meta-classifier for competition with the state of the art. The comparison suggests the meta-classifier is the most promising detection model. In addition, five new Haar features are found to successfully supplement the original Haar feature set. The final comparison of these models will help guide researchers when choosing an appropriate model for rip channel detection, the new Haar features provide researchers with valuable data for detecting rip channels, and the meta-classifier provides a method for increasing the accuracy of a detector through classifier stacking.