A comparative analysis of genetic algorithms and directed grid search for parametric optimization

In this paper, a comparative analysis of the performance of the Genetic Algorithm (GA) and Directed Grid Search (DGS) methods for optimal parametric design is presented. A genetic algorithm is a guided random search mechanism based on the principle of natural selection and population genetics. The Directed Grid Search method uses a selective directed search of grid points in the direction of descent to find the minimum of a real function, when the initial estimate of the location of the minimum and the bounds of the design variables are specified. An experimental comparison and a discussion on the performance of these two methods in solving a set of eight test functions is presented.     

基于种群演化的超参数异步并行搜索

近年来随着深度学习尤其是深度强化学习模型的不断增大,其训练成本即超参数的搜索空间也在不断变大,然而传统超参数搜索算法大部分是基于顺序执行训练,往往需要等待数周甚至数月才有可能找到较优的超参数配置.为解决深度强化学习超参数搜索时间长和难以找到较优超参数配置问题,提出一种新的超参数搜索算法-----基于种群演化的超参数异步并行搜索(PEHS).算法结合演化算法思想,利用固定资源预算异步并行搜索种群模型及其超参数,从而提高算法性能.设计实现在Ray并行分布式框架上运行的参数搜索算法,通过实验表明在并行框架上基于种群演化的超参数异步并行搜索的效果优于传统超参数搜索算法,且性能稳定.     

Prediction of TBM operation parameters using machine learning models based on BPSO

Rational parameters of TBM (Tunnel Boring Machine) are the key to ensuring efficient and safe tunnel construction. Machine learning (ML) has become the main method for predicting operating parameters. Grid Search and optimization algorithms, such as Particle Swarm Optimization (PSO), are often used to find the hyper parameters of ML models but suffer from excessive time and low accuracy. In order to efficiently construct ML models and enhance the accuracy of predicting models, a BPSO (Beetle antennae search Particle Swarm Optimization) algorithm is proposed. Based on the PSO algorithm, the concept of BAS (Beetle Antennae Search) is integrated into the updating process of an individual particle, which improves the random search capability. The convergence of the BPSO algorithm is discussed in terms of inhomogeneous recursive equations and characteristic roots. Then, based on the proposed BPSO prototype, a hybrid ML model BPSO-XGBoost (eXtreme Gradient Boosting) is proposed. We applied the model to the Hangzhou Central Park tunnel project for the prediction of screw conveyer rotational speed. Finally, our model is compared with existing methods. The experimental results show that the BPSO-based model outperforms other traditional ML methods. The BPSO-XGBoost is more accurate than PSO-XGBoost and BPSO-RandomForest for predicting the speed. Also, it is verified that the hyper parameters optimized by the BPSO are better than those optimized by the original PSO. The comprehensive prediction performance ranking of models is as follows: BPSO-XGBoost > PSO-XGBoost > BPSO-RF > PSO-RF. Our models have preferable engineering application value.     

ESTIMATION OF SOLID WASTE BIOMETHANIZATION: DEVELOPMENT OF NOVEL METHODOLOGY AND ANFIS MODELS

This article presents the application of a technique of artificial intelligence (AI) that explores the possibility of using a model to estimate the biomethanization of municipal solid waste (MSW). The model uses data from an experiment in which MSW is anaerobically digested under three different moisture regimes by leachate recycling. A method utilizing a neurofuzzy inference system is used because AI systems have a high capacity for empiric learning.

Considering the importance of finding an effective selection of the most valuable variables for the model, this methodology includes the following techniques: Exhaustive Search (or brute-force search); Stepwise, a step-by-step regression method; and the use of Expert Knowledge. With the use of the fuzzy logic toolbox (MATLAB®), nine models were generated. However, when a case study is used to detail the method, the proposed methodology can also be used with any other system with a set of input and output data.     

基于非负绞杀与长短期记忆神经网络的动态软测量算法

现代工业过程建模中,生产过程的多变量、非线性及动态性会导致模型复杂度增高且建模精度降低.针对这一问题,将非负绞杀算法(NNG)嵌入长短期记忆(LSTM)神经网络,提出一种基于LSTM神经网络及其输入变量选择的动态软测量算法.首先,通过参数优化生成训练好的LSTM神经网络,利用其出色的历史信息记忆能力处理工业过程中的动态、时滞等问题;其次,采用NNG算法对LSTM网络输入权重进行压缩,剔除冗余变量,提高模型精度,并采用网格搜索法与分块交叉验证对其超参数寻优;最后,将算法应用于某火电厂脱硫过程排放烟气SO2浓度软测量建模,并与其它先进算法进行性能比较.实验结果表明所提算法能有效剔除冗余变量,降低模型复杂度并提高其预测性能.     

Gradient-based optimization of hyperparameters

Many machine learning algorithms can be formulated as the minimization of a training criterion that involves a hyperparameter. This hyperparameter is usually chosen by trial and error with a model selection criterion. In this article we present a methodology to optimize several hyperparameters, based on the computation of the gradient of a model selection criterion with respect to the hyperparameters. In the case of a quadratic training criterion, the gradient of the selection criterion with respect to the hyperparameters is efficiently computed by backpropagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyperparameter gradient involving second derivatives of the training criterion.     

Empirical analysis of search based algorithms to identify change prone classes of open source software

There are numerous reasons leading to change in software such as changing requirements, changing technology, increasing customer demands, fixing of defects etc. Thus, identifying and analyzing the change-prone classes of the software during software evolution is gaining wide importance in the field of software engineering. This would help software developers to judiciously allocate the resources used for testing and maintenance. Software metrics can be used for constructing various classification models which can be used for timely identification of change prone classes. Search based algorithms which form a subset of machine learning algorithms can be utilized for constructing prediction models to identify change prone classes of software. Search based algorithms use a fitness function to find the best optimal solution among all the possible solutions. In this work, we analyze the effectiveness of hybridized search based algorithms for change prediction. In other words, the aim of this work is to find whether search based algorithms are capable for accurate model construction to predict change prone classes. We have also constructed models using machine learning techniques and compared the performance of these models with the models constructed using Search Based Algorithms. The validation is carried out on two open source Apache projects, Rave and Commons Math. The results prove the effectiveness of hybridized search based algorithms in predicting change prone classes of software. Thus, they can be utilized by the software developers to produce an efficient and better developed software.     

MultiETSC: automated machine learning for early time series classification

Early time series classification (EarlyTSC) involves the prediction of a class label based on partial observation of a given time series. Most EarlyTSC algorithms consider the trade-off between accuracy and earliness as two competing objectives, using a single dedicated hyperparameter. To obtain insights into this trade-off requires finding a set of non-dominated (Pareto efficient) classifiers. So far, this has been approached through manual hyperparameter tuning. Since the trade-off hyperparameters only provide indirect control over the earliness-accuracy trade-off, manual tuning is tedious and tends to result in many sub-optimal hyperparameter settings. This complicates the search for optimal hyperparameter settings and forms a hurdle for the application of EarlyTSC to real-world problems. To address these issues, we propose an automated approach to hyperparameter tuning and algorithm selection for EarlyTSC, building on developments in the fast-moving research area known as automated machine learning (AutoML). To deal with the challenging task of optimising two conflicting objectives in early time series classification, we propose MultiETSC, a system for multi-objective algorithm selection and hyperparameter optimisation (MO-CASH) for EarlyTSC. MultiETSC can potentially leverage any existing or future EarlyTSC algorithm and produces a set of Pareto optimal algorithm configurations from which a user can choose a posteriori. As an additional benefit, our proposed framework can incorporate and leverage time-series classification algorithms not originally designed for EarlyTSC for improving performance on EarlyTSC; we demonstrate this property using a newly defined, “naïve” fixed-time algorithm. In an extensive empirical evaluation of our new approach on a benchmark of 115 data sets, we show that MultiETSC performs substantially better than baseline methods, ranking highest (avg. rank 1.98) compared to conceptually simpler single-algorithm (2.98) and single-objective alternatives (4.36).

     

Cardiac ScoreCard: A diagnostic multivariate index assay system for predicting a spectrum of cardiovascular disease

Clinical decision support systems (CDSSs) have the potential to save lives and reduce unnecessary costs through early detection and frequent monitoring of both traditional risk factors and novel biomarkers for cardiovascular disease (CVD). However, the widespread adoption of CDSSs for the identification of heart diseases has been limited, likely due to the poor interpretability of clinically relevant results and the lack of seamless integration between measurements and disease predictions. In this paper we present the Cardiac ScoreCard—a multivariate index assay system with the potential to assist in the diagnosis and prognosis of a spectrum of CVD. The Cardiac ScoreCard system is based on lasso logistic regression techniques which utilize both patient demographics and novel biomarker data for the prediction of heart failure (HF) and cardiac wellness. Lasso logistic regression models were trained on a merged clinical dataset comprising 579 patients with 6 traditional risk factors and 14 biomarker measurements. The prediction performance of the Cardiac ScoreCard was assessed with 5-fold cross-validation and compared with reference methods. The experimental results reveal that the ScoreCard models improved performance in discriminating disease versus non-case (AUC = 0.8403 and 0.9412 for cardiac wellness and HF, respectively), and the models exhibit good calibration. Clinical insights to the prediction of HF and cardiac wellness are provided in the form of logistic regression coefficients which suggest that augmenting the traditional risk factors with a multimarker panel spanning a diverse cardiovascular pathophysiology provides improved performance over reference methods. Additionally, a framework is provided for seamless integration with biomarker measurements from point-of-care medical microdevices, and a lasso-based feature selection process is described for the down-selection of biomarkers in multimarker panels.     

Applying models of visual search to map display design

We explored how to use a computational model of visual search to design a map of a mall directory. We parameterized the Guided Search model [Wolfe, J.M., 1994. Guided Search 2.0: a revised model of visual search. Psychonomic Bulletin and Review 1(2), 202–238] for a task involving visual search of a target store in a map. The resulting model was then used to choose color assignments for all the elements of the display that would result in the fastest average search time for the display and search tasks. These predicted optimized color assignments were then tested empirically. The empirical data closely matched the predicted pattern of search times. We conclude that computational models of visual search are sophisticated enough to contribute to the development of optimized map designs, discuss some limitations of the current models, and suggest directions for further development.     

秃鹰搜索算法的优化及图像分类应用

针对秃鹰搜索算法(BES)存在求解的稳定性差且准确性低, 鲁棒性差等缺点, 提出了一种基于秃鹰搜索算法的新型算法(NBES). 首先, 在BES算法的选择搜索空间阶段融合正余弦优化机制算法, 构建融合后的位置更新公式. 其次, 在BES算法的搜索空间猎物阶段加入惯性权重自适应位置更新策略. 最后, 在BES算法俯冲阶段融合萤火虫优化机制算法, 重新定义位置更新公式. 通过11个标准测试函数验证NBES算法性能, 实验表明, NBES算法寻优准确性、收敛速度、鲁棒性都优于BES算法. 为了验证新算法的实际应用价值, 利用NBES算法优化卷积神经网络(CNN)中的超参数学习率, 并将优化后的图像分类模型用于医学影像病理性分类预测, 实验表明, 经过优化的CNN模型分类精度提高9%.     

Automatic identification of cardiac health using modeling techniques: A comparative study

Heart rate variability (HRV), a widely adopted quantitative marker of the autonomic nervous system can be used as a predictor of risk of cardiovascular diseases. Moreover, decreased heart rate variability (HRV) has been associated with an increased risk of cardiovascular diseases. Hence in this work HRV signal is used as the base signal for predicting the risk of cardiovascular diseases. The present study concerns nine cardiac classes that include normal sinus rhythm (NSR), congestive heart failure (CHF), atrial fibrillation (AF), ventricular fibrillation (VF), preventricular contraction (PVC), left bundle branch block (LBBB), complete heart block (CHB), ischemic/dilated cardiomyopathy (ISCH) and sick sinus syndrome (SSS). A total of 352 cardiac subjects belonging to the nine classes were analyzed in the frequency domain. The fast Fourier transforms (FFT) and three other modeling techniques namely, autoregressive (AR) model, moving average (MA) model and the autoregressive moving average (ARMA) model are used to estimate the power spectral densities of the RR interval variability. The spectral parameters obtained from the spectral analysis of the HRV signals are used as the input parameters to the artificial neural network (ANN) for classification of the different cardiac classes. Our findings reveal that the ARMA modeling technique seems to give better resolution and would be more promising for clinical diagnosis.     

基于MPSO-SVM巷道围岩松动圈预测研究

针对目前巷道围岩松动圈确定方法的种种缺陷,提出了一种新的预测方法,采用改进的粒子群算法（MPSO）优化支持向量机（SVM）对巷道围岩松动圈进行预测。在标准PSO中引入压缩因子,实现了算法全局搜索和局部寻优的有效平衡;应用MPSO对SVM的参数C和g进行优化,建立MPSO-SVM回归预测模型;将该预测模型应用于巷道围岩松动圈的预测,将预测性能与PSO-SVM、GA（遗传算法）-SVM、GSM（网格搜索）-SVM模型、BP神经网络进行对比分析。结果表明：该模型具有较强的泛化能力,较高的预测精度,可以对围岩松动圈厚度进行有效预测。     

基于EDA-DP的LS-SVM参数优化选择

通过分析最小二乘支持向量机(LS-SVM)模型的超参数选择对分类器的影响,提出一种采用多样性保持的分布估计算法(EDA-DP)优化选择LS-SVM模型参数的方法。使用基于EDA-DP的LS-SVM分类器模型对基准数据集和雷达目标高分辨距离像数据集进行仿真实验,结果表明,该模型相比基于网格法的分类器模型,平均识别率分别提高了4.2%和1.76%,具有更好的分类性能和泛化能力。     

Intelligent Intrusion Detection System for Industrial Internet of Things Environment

Rapid increase in the large quantity of industrial data, Industry 4.0/5.0 poses several challenging issues such as heterogeneous data generation, data sensing and collection, real-time data processing, and high request arrival rates. The classical intrusion detection system (IDS) is not a practical solution to the Industry 4.0 environment owing to the resource limitations and complexity. To resolve these issues, this paper designs a new Chaotic Cuckoo Search Optimization Algorithm (CCSOA) with optimal wavelet kernel extreme learning machine (OWKELM) named CCSOA-OWKELM technique for IDS on the Industry 4.0 platform. The CCSOA-OWKELM technique focuses on the design of feature selection with classification approach to achieve minimum computation complexity and maximum detection accuracy. The CCSOA-OWKELM technique involves the design of CCSOA based feature selection technique, which incorporates the concepts of chaotic maps with CSOA. Besides, the OWKELM technique is applied for the intrusion detection and classification process. In addition, the OWKELM technique is derived by the hyperparameter tuning of the WKELM technique by the use of sunflower optimization (SFO) algorithm. The utilization of CCSOA for feature subset selection and SFO algorithm based hyperparameter tuning leads to better performance. In order to guarantee the supreme performance of the CCSOA-OWKELM technique, a wide range of experiments take place on two benchmark datasets and the experimental outcomes demonstrate the promising performance of the CCSOA-OWKELM technique over the recent state of art techniques.     

A new and fast rival genetic algorithm for feature selection

Feature selection is one of the significant steps in classification tasks. It is a pre-processing step to select a small subset of significant features that can contribute the most to the classification process. Presently, many metaheuristic optimization algorithms were successfully applied for feature selection. The genetic algorithm (GA) as a fundamental optimization tool has been widely used in feature selection tasks. However, GA suffers from the hyperparameter setting, high computational complexity, and the randomness of selection operation. Therefore, we propose a new rival genetic algorithm, as well as a fast version of rival genetic algorithm, to enhance the performance of GA in feature selection. The proposed approaches utilize the competition strategy that combines the new selection and crossover schemes, which aim to improve the global search capability. Moreover, a dynamic mutation rate is proposed to enhance the search behaviour of the algorithm in the mutation process. The proposed approaches are validated on 23 benchmark datasets collected from the UCI machine learning repository and Arizona State University. In comparison with other competitors, proposed approach can provide highly competing results and overtake other algorithms in feature selection.

     

Using Local Popularity of Web Resources for Geo-Ranking of Search Engine Results

Search engines retrieve and rank Web pages which are not only relevant to a query but also important or popular for the users. This popularity has been studied by analysis of the links between Web resources. Link-based page ranking models such as PageRank and HITS assign a global weight to each page regardless of its location. This popularity measurement has shown successful on general search engines. However unlike general search engines, location-based search engines should retrieve and rank higher the pages which are more popular locally. The best results for a location-based query are those which are not only relevant to the topic but also popular with or cited by local users. Current ranking models are often less effective for these queries since they are unable to estimate the local popularity. We offer a model for calculating the local popularity of Web resources using back link locations. Our model automatically assigns correct locations to the links and content and uses them to calculate new geo-rank scores for each page. The experiments show more accurate geo-ranking of search engine results when this model is used for processing location-based queries.     

Standard versus uniform binary search and their variants in learned static indexing: The case of the searching on sorted data benchmarking software platform

Learned Indexes use a model to restrict the search of a sorted table to a smaller interval. Typically, a final binary search is done using the lower_bound routine of the Standard C++ library. Recent studies have shown that on current processors other search approaches (such as k-ary search) can be more efficient in some applications. Using the SOSD learned indexing benchmarking software, we extend these results to show that k-ary search is indeed a better choice when using learned indexes. We highlight how such a choice may be dependent on the computer architecture used, for example, Intel I7 or Apple M1, and provide guidelines for the selection of the Search routine within the learned indexing framework.     

Asymptotic behaviors of support vector machines with Gaussian kernel

Support vector machines (SVMs) with the gaussian (RBF) kernel have been popular for practical use. Model selection in this class of SVMs involves two hyperparameters: the penalty parameter C and the kernel width sigma. This letter analyzes the behavior of the SVM classifier when these hyperparameters take very small or very large values. Our results help in understanding the hyperparameter space that leads to an efficient heuristic method of searching for hyperparameter values with small generalization errors. The analysis also indicates that if complete model selection using the gaussian kernel has been conducted, there is no need to consider linear SVM.