Prediction of performance and emission characteristics in a biodiesel engine using WCO ester: a comparative study of neural networks

In this study, the applicability of Artificial Neural Networks (ANNs) has been investigated for predicting the performance and emission characteristics of a diesel engine fuelled with Waste cooking oil (WCO). ANN modeling was done using multilayer perception (MLP) and radial basis functions (RBF). In the radial basis functions, centers were initialized by two different methods namely random selection method and using clustering algorithm. In the clustering method, center initialization was done using FCM (Fuzzy \(c\) means) and CDWFCM (cluster dependent weighted fuzzy \(c\) means) algorithms. The networks were trained using the experimental data, wherein load percentage, compression ratio, blend percentage, injection timing and injection pressure were taken as the input parameters and brake thermal efficiency, brake specific energy consumption, exhaust gas temperature and engine emissions were used as the output parameters. The investigation showed that ANN predicted results matched well with the experimental results over a wide range of operating conditions for both models. A comparison was made between ANN models and regression models. ANN performed better than the regression models. Similarly a comparison of MLP and RBF indicated that RBF with CDWFCM performed better than MLP networks with lower Mean Relative Error (MRE) and higher accuracy of prediction.     

Covariance matrix self-adaptation evolution strategies and other metaheuristic techniques for neural adaptive learning

A covariance matrix self-adaptation evolution strategy (CMSA-ES) was compared with several metaheuristic techniques for multilayer perceptron (MLP)-based function approximation and classification. Function approximation was based on simulations of several 2D functions and classification analysis was based on nine cancer DNA microarray data sets. Connection weight learning by MLPs was carried out using genetic algorithms (GA?CMLP), covariance matrix self-adaptation-evolution strategies (CMSA-ES?CMLP), back-propagation gradient-based learning (MLP), particle swarm optimization (PSO?CMLP), and ant colony optimization (ACO?CMLP). During function approximation runs, input-side activation functions evaluated included linear, logistic, tanh, Hermite, Laguerre, exponential, and radial basis functions, while the output-side function was always linear. For classification, the input-side activation function was always logistic, while the output-side function was always regularized softmax. Self-organizing maps and unsupervised neural gas were used to reduce dimensions of original gene expression input features used in classification. Results indicate that for function approximation, use of Hermite polynomials for activation functions at hidden nodes with CMSA-ES?CMLP connection weight learning resulted in the greatest fitness levels. On average, the most elite chromosomes were observed for MLP ( ${\rm MSE}=0.4977$ ), CMSA-ES?CMLP (0.6484), PSO?CMLP (0.7472), ACO?CMLP (1.3471), and GA?CMLP (1.4845). For classification analysis, overall average performance of classifiers used was 92.64% (CMSA-ES?CMLP), 92.22% (PSO?CMLP), 91.30% (ACO?CMLP), 89.36% (MLP), and 60.72% (GA?CMLP). We have shown that a reliable approach to function approximation can be achieved through application of MLP connection weight learning when the assumed function is unknown. In this scenario, the MLP architecture itself defines the equation used for solving the unknown parameters relating input and output target values. A major drawback of implementing CMSA-ES into an MLP is that when the number of MLP weights is large, the ${{\mathcal{O}}}(N^3)$ Cholesky factorization becomes a bottleneck for performance. As an alternative, feature reduction using SOM and NG can greatly enhance performance of CMSA-ES?CMLP by reducing $N.$ Future research into the speeding up of Cholesky factorization for CMSA-ES will be helpful in overcoming time complexity problems related to a large number of connection weights.     

Exploring comprehensible classification rules from trained neural networks integrated with a time-varying binary particle swarm optimizer

Purpose

Extracting comprehensible classification rules is the most emphasized concept in data mining researches. In order to obtain accurate and comprehensible classification rules from databases, a new approach is proposed by combining advantages of artificial neural networks (ANN) and swarm intelligence.

Method

Artificial neural networks (ANNs) are a group of very powerful tools applied to prediction, classification and clustering in different domains. The main disadvantage of this general purpose tool is the difficulties in its interpretability and comprehensibility. In order to eliminate these disadvantages, a novel approach is developed to uncover and decode the information hidden in the black-box structure of ANNs. Therefore, in this paper a study on knowledge extraction from trained ANNs for classification problems is carried out. The proposed approach makes use of particle swarm optimization (PSO) algorithm to transform the behaviors of trained ANNs into accurate and comprehensible classification rules. Particle swarm optimization with time varying inertia weight and acceleration coefficients is designed to explore the best attribute-value combination via optimizing ANN output function.

Results

The weights hidden in trained ANNs turned into comprehensible classification rule set with higher testing accuracy rates compared to traditional rule based classifiers.     

Robust learning algorithm for multiplicative neuron model artificial neural networks

The two most commonly used types of artificial neural networks (ANNs) are the multilayer feed-forward and multiplicative neuron model ANNs. In the literature, although there is a robust learning algorithm for the former, there is no such algorithm for the latter. Because of its multiplicative structure, the performance of multiplicative neuron model ANNs is affected negatively when the dataset has outliers. On this issue, a robust learning algorithm for the multiplicative neuron model ANNs is proposed that uses Huber's loss function as fitness function. The training of the multiplicative neuron model is performed using particle swarm optimization. One principle advantage of this algorithm is that the parameter of the scale estimator, which is an important factor affecting the value of Huber's loss function, is also estimated with the proposed algorithm. To evaluate the performance of the proposed method, it is applied to two well-known real world time series datasets, and also a simulation study is performed. The algorithm has superior performance both when it is applied to real world time series datasets and the simulation study when compared with other ANNs reported in the literature. Another of its advantages is that, for datasets with outliers, the results are very close to the results obtained from the original datasets. In other words, we demonstrate that the algorithm is unaffected by outliers and has a robust structure.     

Fast and Stable Learning Utilizing Singular Regions of Multilayer Perceptron

In the parameter space of MLP(J), multilayer perceptron with J hidden units, there exist flat areas called singular regions created by applying reducibility mappings to the optimal solution of MLP( $J-1$ ). Since such singular regions cause serious stagnation of learning, a learning method to avoid singular regions has been desired. However, such avoiding does not guarantee the quality of the final solutions. This paper proposes a new learning method which does not avoid but makes good use of singular regions to stably and successively find excellent solutions commensurate with MLP(J). The proposed method worked well in our experiments using artificial and real data sets.     

Spectrum occupancy prediction based on functional link artificial neural network (FLANN) in ISM band

Recent advancements in artificial neural networks (ANNs) motivated us to design a simple and faster spectrum prediction model termed the functional link artificial neural network (FLANN). The main objective of this paper is to gather realistic data to obtain utilization statistics for the industrial, scientific and medical band of 2.4–2.5 GHz. To present the occupancy statistics, we conducted measurement in indoors at the Swearingen Engineering Center, University of South Carolina. Further, we introduce different threshold-based spectrum prediction schemes to show the impact of threshold on occupancy, and propose a spectrum prediction algorithm based on FLANN to forecast a future spectrum usage profile from historical occupancy statistics. Spectrum occupancy is estimated and predicted by employing different ANN models including the Feed-forward multilayer perceptron (MLP), Recurrent MLP, Chebyshev FLANN and Trigonometric FLANN. It is observed that the absence of a hidden layer in FLANN makes it more efficient than the MLP model in predicting the occupancy faster and with less complexity. A set of illustrative results are presented to validate the performance of our proposed learning scheme.

     

Topographic Bernoulli block mixture mapping for binary tables

Co-clustering methods are valuable parsimonious approaches for the analysis of a binary data table by a simultaneous partitioning of the rows or the columns. Bringing the property of visualization to co-clustering is of first importance for a fast access to the essential topics and their relations. We propose a new generative self-organizing map by a particular parameterization of the Bernoulli block mixture model. The method is called block GTM or topographic block model. Thanks to the underlying probabilistic framework, the inference of the parameters of the method is performed with the block EM algorithm. At the maximization step, two local quadratic approximations of the objective function arise from a second-order optimization, respectively, with the Newton–Raphson algorithm and with a variational bound of the sigmoid function. In the experiments with several datasets, the two algorithms are able to outperform former approaches and lead to similar results when the parameters are regularized with a \(L_1\) -norm. The conclusion summarizes the contribution and some perspectives.     

NeVer: a tool for artificial neural networks verification

The adoption of Artificial Neural Networks (ANNs) in safety-related applications is often avoided because it is difficult to rule out possible misbehaviors with traditional analytical or probabilistic techniques. In this paper we present NeVer, our tool for checking safety of ANNs. NeVer encodes the problem of verifying safety of ANNs into the problem of satisfying corresponding Boolean combinations of linear arithmetic constraints. We describe the main verification algorithm and the structure of NeVer. We present also empirical results confirming the effectiveness of NeVer on realistic case studies.     

Proposing two new metaheuristic algorithms of ALO-MLP and SHO-MLP in predicting bearing capacity of circular footing located on horizontal multilayer soil

In this study, for the issue of shallow circular footing’s bearing capacity (also shown as F_ult), we used the merits of artificial neural network (ANN), while optimized it by two metaheuristic algorithms (i.e., ant lion optimization (ALO) and the spotted hyena optimizer (SHO)). Several studies demonstrated that ANNs have significant results in terms of predicting the soil’s bearing capacity. Nevertheless, most models of ANN learning consist of different disadvantages. Accordantly, we focused on the application of two hybrid models of ALO–MLP and SHO–MLP for predicting the F_ult placed in layered soils. Moreover, we performed an Extensive Finite Element (FE) modeling on 16 sets of soil layer (soft soil placed onto stronger soil and vice versa) considering a database that consists of 703 testing and 2810 training datasets for preparing the training and testing datasets. The independent variables in terms of ALO and SHO algorithms have been optimized by taking into account a trial and error process. The input data layers consisted of (i) upper layer foundation/thickness width (h/B) ratio, (ii) bottom and topsoil layer properties (for example, six of the most important properties of soil), (iii) vertical settlement (s), (iv) footing width (B), where the main target was taken F_ult. According to RMSE and R², values of (0.996 and 0.034) and (0.994 and 0.044) are obtained for training dataset and values of (0.994 and 0.040) and (0.991 and 0.050) are found for the testing dataset of proposed SHO–MLP and ALO–MLP best-fit prediction network structures, respectively. This proves higher reliability of the proposed hybrid model of SHO–MLP in approximating shallow circular footing bearing capacity.

     

Optimized Parameter Search for Large Datasets of the Regularization Parameter and Feature Selection for Ridge Regression

In this paper we propose mathematical optimizations to select the optimal regularization parameter for ridge regression using cross-validation. The resulting algorithm is suited for large datasets and the computational cost does not depend on the size of the training set. We extend this algorithm to forward or backward feature selection in which the optimal regularization parameter is selected for each possible feature set. These feature selection algorithms yield solutions with a sparse weight matrix using a quadratic cost on the norm of the weights. A naive approach to optimizing the ridge regression parameter has a computational complexity of the order $O(R K N^{2} M)$ with $R$ the number of applied regularization parameters, $K$ the number of folds in the validation set, $N$ the number of input features and $M$ the number of data samples in the training set. Our implementation has a computational complexity of the order $O(KN^3)$ . This computational cost is smaller than that of regression without regularization $O(N^2M)$ for large datasets and is independent of the number of applied regularization parameters and the size of the training set. Combined with a feature selection algorithm the algorithm is of complexity $O(RKNN_s^3)$ and $O(RKN^3N_r)$ for forward and backward feature selection respectively, with $N_s$ the number of selected features and $N_r$ the number of removed features. This is an order $M$ faster than $O(RKNN_s^3M)$ and $O(RKN^3N_rM)$ for the naive implementation, with $N \ll M$ for large datasets. To show the performance and reduction in computational cost, we apply this technique to train recurrent neural networks using the reservoir computing approach, windowed ridge regression, least-squares support vector machines (LS-SVMs) in primal space using the fixed-size LS-SVM approximation and extreme learning machines.     

A new hybrid combinatorial genetic algorithm for multidimensional knapsack problems

Multidimensional knapsack problem (MKP) is known to be a NP-hard problem, more specifically a NP-complete problem, which cannot be resolved in polynomial time up to now. MKP can be applicable in many management, industry and engineering fields, such as cargo loading, capital budgeting and resource allocation, etc. In this article, using a combinational permutation constructed by the convex combinatorial value \(M_j=(1-\lambda ) u_j+ \lambda x^\mathrm{LP}_j\) of both the pseudo-utility ratios of MKP and the optimal solution \(x^\mathrm{LP}\) of relaxed LP, we present a new hybrid combinatorial genetic algorithm (HCGA) to address multidimensional knapsack problems. Comparing to Chu’s GA (J Heuristics 4:63–86, 1998), empirical results show that our new heuristic algorithm HCGA obtains better solutions over 270 standard test problem instances.     

Prediction and optimization of electrospinning parameters for polymethyl methacrylate nanofiber fabrication using response surface methodology and artificial neural networks

Since the fiber diameter determines the mechanical, electrical, and optical properties of electrospun nanofiber mats, the effect of material and process parameters on electrospun polymethyl methacrylate (PMMA) fiber diameter were studied. Accordingly, the prediction and optimization of input factors were performed using the response surface methodology (RSM) with the design of experiments technique and artificial neural networks (ANNs). A central composite design of RSM was employed to develop a mathematical model as well as to define the optimum condition. A three-layered feed-forward ANN model was designed and used for the prediction of the response factor, namely the PMMA fiber diameter (in nm). The parameters studied were polymer concentration (13–28 wt%), feed rate (1–5 mL/h), and tip-to-collector distance (10–23 cm). From the analysis of variance, the most significant factor that caused a remarkable impact on the experimental design response was identified. The predicted responses using the RSM and ANNs were compared in figures and tables. In general, the ANNs outperformed the RSM in terms of accuracy and prediction of obtained results.     

An efficient model based on artificial bee colony optimization algorithm with Neural Networks for electric load forecasting

Short-term electric load forecasting (STLF) is an essential tool for power generation planning, transmission dispatching, and day-to-day utility operations. A number of techniques are used and reported in the literature to build an accurate forecasting model. Out of them Artificial Neural Networks (ANN) are proven most promising technique for STLF model building. Many learning schemes are being used to boost the ANN performance with improved results. This motivated us to explore better optimization approaches to devise a more suitable prediction technique. In this study, we propose a new hybrid model for STLF by combining greater optimization ability of artificial bee colony (ABC) algorithm with ANN. The ABC is used as an alternative learning scheme to get optimized set of neuron connection weights for ANN. This formulation showed improved convergence rate without trapping into local minimum. Forecasting results obtained by this new approach have been presented and compared with other mature and competitive approaches, which confirms its applicability in forecasting domain.     

Sequence Input-Based Quantum-Inspired Neural Networks with Applications

To enhance the approximation and generalization ability of artificial neural networks (ANNs) by employing the principle of quantum rotation gate and controlled-not gate, a quantum-inspired neuron with sequence input is proposed. In the proposed model, the discrete sequence input is represented by the qubits, which, as the control qubits of the controlled-not gate after being rotated by the quantum rotation gates, control the target qubit for reverse. The model output is described by the probability amplitude of state \(|1\rangle \) in the target qubit. Then a quantum-inspired neural networks (QINN) is designed by employing the quantum-inspired neurons to the hidden layer and the common neurons to the output layer. The algorithm of QINN is derived by employing the Levenberg–Marquardt algorithm. Simulation results of some benchmark problems show that, under a certain condition, the QINN is obviously superior to the classical ANN.     

A hybrid clustering algorithm based on PSO with dynamic crossover

In order to overcome the premature convergence in particle swarm optimization (PSO), we introduce dynamical crossover, a crossover operator with variable lengths and positions, to PSO, which is briefly denoted as CPSO. To get rid of the drawbacks of only finding the convex clusters and being sensitive to the initial points in $k$ -means algorithm, a hybrid clustering algorithm based on CPSO is proposed. The difference between the work and the existing ones lies in that CPSO is firstly introduced into $k$ -means. Experimental results performing on several data sets illustrate that the proposed clustering algorithm can get completely rid of the shortcomings of $k$ -means algorithms, and acquire correct clustering results. The application in image segmentation illustrates that the proposed algorithm gains good performance.     

A note on the growth factor in Gaussian elimination for accretive-dissipative matrices

This short note proves that if \(A\) is accretive-dissipative, then the growth factor for such \(A\) in Gaussian elimination is less than \(4\) . If \(A\) is a Higham matrix, i.e., the accretive-dissipative matrix \(A\) is complex symmetric, then the growth factor is less than \(2\sqrt{2}\) . The result obtained improves those of George et al. in [Numer. Linear Algebra Appl. 9, 107–114 (2002)] and is one step closer to the final solution of Higham’s conjecture.     

Super-EGO: fast multi-dimensional similarity join

Efficient processing of high-dimensional similarity joins plays an important role for a wide variety of data-driven applications. In this paper, we consider $\varepsilon $ -join variant of the problem. Given two $d$ -dimensional datasets and parameter $\varepsilon $ , the task is to find all pairs of points, one from each dataset that are within $\varepsilon $ distance from each other. We propose a new $\varepsilon $ -join algorithm, called Super-EGO, which belongs the EGO family of join algorithms. The new algorithm gains its advantage by using novel data-driven dimensionality re-ordering technique, developing a new EGO-strategy that more aggressively avoids unnecessary computation, as well as by developing a parallel version of the algorithm. We study the newly proposed Super-EGO algorithm on large real and synthetic datasets. The empirical study demonstrates significant advantage of the proposed solution over the existing state of the art techniques.     

The Minimum Vulnerability Problem

We revisit the problem of finding \(k\) paths with a minimum number of shared edges between two vertices of a graph. An edge is called shared if it is used in more than one of the \(k\) paths. We provide a \({\lfloor {k/2}\rfloor }\) -approximation algorithm for this problem, improving the best previous approximation factor of \(k-1\) . We also provide the first approximation algorithm for the problem with a sublinear approximation factor of \(O(n^{3/4})\) , where \(n\) is the number of vertices in the input graph. For sparse graphs, such as bounded-degree and planar graphs, we show that the approximation factor of our algorithm can be improved to \(O(\sqrt{n})\) . While the problem is NP-hard, and even hard to approximate to within an \(O(\log n)\) factor, we show that the problem is polynomially solvable when \(k\) is a constant. This settles an open problem posed by Omran et al. regarding the complexity of the problem for small values of \(k\) . We present most of our results in a more general form where each edge of the graph has a sharing cost and a sharing capacity, and there is a vulnerability parameter \(r\) that determines the number of times an edge can be used among different paths before it is counted as a shared/vulnerable edge.     

Fuzzy learning vector quantization for hyperspectral coastal vegetation classification

Artificial neural networks (ANNs) may be of significant value in extracting vegetation type information in complex vegetation mapping problems, particularly in coastal wetland environments. Unsupervised, self-organizing ANNs have not been employed as frequently as supervised ANNs for vegetation mapping tasks, and further remote sensing research involving fuzzy ANNs is also needed. In this research, the utility of a fuzzy unsupervised ANN, specifically a fuzzy learning vector quantization (FLVQ) ANN, was investigated in the context of hyperspectral AVIRIS image classification. One key feature of the neural approach is that unlike conventional hyperspectral data processing methods, endmembers for a given scene, which can be difficult to determine with confidence, are not required for neural analysis. The classification accuracy of FLVQ was comparable to a conventional supervised multi-layer perceptron, trained with backpropagation (MLP) (KHAT (Kˆ) accuracy: 82.82% and 84.66%, respectively; normalized accuracy: 74.60% and 75.85%, respectively), with no significant difference at the 95% confidence level. All neural algorithms in the experiment yielded significantly higher classification accuracies than the conventional endmember-based hyperspectral mapping method assessed (i.e., matched filtering, where Kˆ accuracy = 61.00% and normalized accuracy = 57.96%). FLVQ was also dramatically more computationally efficient than the baseline supervised and unsupervised ANN algorithms tested, including the MLP and the Kohonen self-organizing map (SOM), respectively. The 400-neuron FLVQ network required only 3.6% of the computation time used by the MLP network, and only 5.9% of the MLP time was used by the 588-neuron FLVQ network. In addition, the 400-neuron FLVQ used only 16.7% of the time used by the 400-neuron SOM for model development.