A cortex-like learning machine for temporal hierarchical pattern clustering, detection, and recognition

A learning machine, called a clustering interpreting probabilistic associative memory (CIPAM), is proposed. CIPAM consists of a clusterer and an interpreter. The clusterer is a recurrent hierarchical neural network of unsupervised processing units (UPUs). The interpreter is a number of supervised processing units (SPUs) that branch out from the clusterer. Each processing unit (PU), UPU or SPU, comprises “dendritic encoders” for encoding inputs to the PU, “synapses” for storing resultant codes, a “nonspiking neuron” for generating inhibitory graded signals to modulate neighboring spiking neurons, “spiking neurons” for computing the subjective probability distribution (SPD) or the membership function, in the sense of fuzzy logic, of the label of said inputs to the PU and generating spike trains with the SPD or membership function as the firing rates, and a masking matrix for maximizing generalization. While UPUs employ unsupervised covariance learning mechanisms, SPUs employ supervised ones. They both also have unsupervised accumulation learning mechanisms. The clusterer of CIPAM clusters temporal and spatial data. The interpreter interprets the resultant clusters, effecting detection and recognition of temporal and hierarchical causes.     

The Architecture and Performance of a Stochastic Competitive Evolutionary Neural Tree Network

A new dynamic tree structured network—the Stochastic Competitive Evolutionary Neural Tree (SCENT) is introduced. The network is able to provide a hierarchical classification of unlabelled data sets. The main advantage that SCENT offers over other hierarchical competitive networks is its ability to self-determine the number and structure of the competitive nodes in the network without the need for externally set parameters. The network produces stable classificatory structures by halting its growth using locally calculated, stochastically controlled, heuristics. The performance of the network is analysed by comparing its results with that of a good non-hierarchical clusterer, and with three other hierarchical clusterers and its non stochastic predecessor. SCENT's classificatory capabilities are demonstrated by its ability to produce a representative hierarchical structure to classify a broad range of data sets.     

基于Bagging的选择性聚类集成

使用集成学习技术来提高聚类性能.由于聚类使用的训练样本缺乏期望输出,与监督学习下的集成相比,在对个体学习器进行结合时更加困难.通过对不同的聚类结果进行配准,并基于互信息权进行个体学习器的选择,提出了基于Bagging的选择性聚类集成算法.实验表明,该算法能够有效地改善聚类结果.     

Multidimensional urban segregation: toward a neural network measure

We introduce a multidimensional, neural network approach to reveal and measure urban segregation phenomena, based on the self-organizing map algorithm (SOM). The multidimensionality of SOM allows one to apprehend a large number of variables simultaneously, defined on census blocks or other types of statistical blocks, and to perform clustering along them. Levels of segregation are then measured through correlations between distances on the neural network and distances on the actual geographical map. Further, the stochasticity of SOM enables one to quantify levels of heterogeneity across census blocks. We illustrate this new method on data available for the city of Paris.

     

Neural network ensemble operators for time series forecasting

The combination of forecasts resulting from an ensemble of neural networks has been shown to outperform the use of a single “best” network model. This is supported by an extensive body of literature, which shows that combining generally leads to improvements in forecasting accuracy and robustness, and that using the mean operator often outperforms more complex methods of combining forecasts. This paper proposes a mode ensemble operator based on kernel density estimation, which unlike the mean operator is insensitive to outliers and deviations from normality, and unlike the median operator does not require symmetric distributions. The three operators are compared empirically and the proposed mode ensemble operator is found to produce the most accurate forecasts, followed by the median, while the mean has relatively poor performance. The findings suggest that the mode operator should be considered as an alternative to the mean and median operators in forecasting applications. Experiments indicate that mode ensembles are useful in automating neural network models across a large number of time series, overcoming issues of uncertainty associated with data sampling, the stochasticity of neural network training, and the distribution of the forecasts.     

A neural network-based approach for dynamic quality prediction in a plastic injection molding process

This paper presents an innovative neural network-based quality prediction system for a plastic injection molding process. A self-organizing map plus a back-propagation neural network (SOM-BPNN) model is proposed for creating a dynamic quality predictor. Three SOM-based dynamic extraction parameters with six manufacturing process parameters and one level of product quality were dedicated to training and testing the proposed system. In addition, Taguchi’s parameter design method was also applied to enhance the neural network performance. For comparison, an additional back-propagation neural network (BPNN) model was constructed for which six process parameters were used for training and testing. The training and testing data for the two models respectively consisted of 120 and 40 samples. Experimental results showed that such a SOM-BPNN-based model can accurately predict the product quality (weight) and can likely be used for various practical applications.     

Milling force prediction using regression and neural networks

This study focuses on developing a good empirical relationship between the cutting force in an end milling operation and the cutting parameters such as speed, feed and depth-of-cut, by using both multiple regression and neural network modeling processes. A regression model was first fitted to experimentally collected data and any abnormal data points indicated by this analysis were filtered out. By repeating this process several times, a final set of filtered data was obtained and analyzed using neural networks to yield a good, final model. This study shows that analyzing milling force data using conventional regression can lead to a more accurate neural networks model for force prediction.     

大黄素衍生物抗肿瘤活性的神经网络模型

目的:建立大黄素衍生物抗肿瘤活性的神经网络模型。方法:采用量子化学的AM1算法,计算了12个大黄素衍生物分子的结构参数,并用逐步回归分析,筛选结构参数。结果:利用筛选后的结构参数,建立大黄素衍生物抗肿瘤活性的神经网络模型。结论:抽一法交叉预报结果表明,本文建立的大黄素衍生物抗肿瘤活性的神经网络模型,预报结果可靠,具有一定的应用价值。     

Use of hybrid intelligent computing in mineral resources evaluation

Mineral resources are a formal quantification of naturally occurring materials. Estimation of resource parameters such as grade and thickness may be carried out using different methodologies. In this paper, a soft methodology, which is artificial neural network (ANN) based fuzzy modelling is presented for grade estimation and its stages are demonstrated. The neuro-fuzzy method uses preliminary clustering and finally estimates the ore grades based on radial basis neural network and interpolation. Two case studies designed for both simulated and real data sets indicate that the approach is relatively accurate and flexible. In addition, the method is suitable for modelling via limited number of data. The results and performance comparisons with conventional methods show that the computing method is efficient.     

MGA-Net+: Acceleration-based packaging work recognition using motif-guided attention networks

This study presents a new method for recognizing complex human activities within the logistics domain, such as packaging operations, using acceleration data from a body-worn sensor. The recognition of packaging tasks using standard supervised machine learning is complex because the observed data vary considerably depending on the number of items to be packed, the size of the items, and other parameters. In this study, we focused on the characteristics and necessary key actions (motions) that occur during a specific operation. For instance, when the packaging tape is stretched while assembling the shipping boxes. To focus on these characteristic actions when recognizing data, we propose the use of an attention-based neural network. With our method, the attention-based neural network’s training is guided such that its focus is on the motifs. In addition, this method was designed to accurately recognize short operations by leveraging data augmentation techniques. We tested our method on two logistics datasets and achieved a 3.9% improvement over the previous MGA-Net approach.     

基于降噪自编码神经网络的事件相关电位脑电信号分析方法

提出一种基于降噪自编码神经网络事件相关电位分析方法,首先建立3层神经网络结构,利用降噪自编码对神经网络进行初始化,实现了降噪自编码深度学习模型的无监督学习.从无标签数据中自动学习数据特征,通过优化模型训练得到的权值作为神经网络初始化参数.其次,经过有标签的样本进行网络参数的微调即可完成对神经网络的训练,该方法有效解决了神经网络训练中因随机选择初始化参数,而导致网络易陷入局部极小的缺陷.最后,利用上述神经网络对第3届脑机接口竞赛数据集Data set Ⅱ(事件相关电位脑电信号)进行分类分析.实验结果表明:利用降噪自编码迭代2500次训练神经网络模型,在受试者A和受试者B样本数据叠加5次、10次、15次3种情况下获得的分类准确率分别为73.4%, 87.4%和97.2%.该最高准确率优于其他分类方法,比竞赛第1名联合支持向量机(SVM)分类器(ESVM)提高了0.7%,为事件相关电位脑电信号提供了一种深度学习分析方法.     

Assessment and prediction of tropospheric ozone concentration levels using artificial neural networks

This work deals specifically with the use of a neural network for ozone modelling in the lower atmosphere. The development of a neural network model is presented to predict the tropospheric (surface or ground) ozone concentrations as a function of meteorological conditions and various air quality parameters. The development of the model was based on the realization that the prediction of ozone from a theoretical basis (i.e. detailed atmospheric diffusion model) is difficult. In contrast, neural networks are useful for modelling because of their ability to be trained using historical data and because of their capability for modelling highly non-linear relationships. The network was trained using summer meteorological and air quality data when the ozone concentrations are the highest. The data were collected from an urban atmosphere. The site was selected to represent a typical residential area with high traffic influences. Three neural network models were developed. The main emphasis of the first model has been placed on studying the factors that control the ozone concentrations during a 24-hour period (daylight and night hours were included). The second model was developed to study the factors that regulate the ozone concentrations during daylight hours at which higher concentrations of ozone were recorded. The third model was developed to predict daily maximum ozone levels. The predictions of the models were found to be consistent with observations. A partitioning method of the connection weights of the network was used to study the relative percent contribution of each of the input variables. The contribution of meteorology on the ozone concentration variation was found to fall within the range 33.15–40.64%. It was also found that nitrogen oxide, sulfur dioxide, relative humidity, non-methane hydrocarbon and nitrogen dioxide have the most effect on the predicted ozone concentrations. In addition, temperature played an important role while solar radiation had a lower effect than expected. The results of this study indicate that the artificial neural network (ANN) is a promising method for air pollution modelling.     

Prediction of the response under impact of steel armours using a multilayer perceptron

This article puts forward the results obtained when using a neural network as an alternative to classical methods (simulation and experimental testing) in the prediction of the behaviour of steel armours against high-speed impacts. In a first phase, a number of impact cases are randomly generated, varying the values of the parameters which define the impact problem (radius, length and velocity of the projectile; thickness of the protection). After simulation of each case using a finite element code, the above-mentioned parameters and the results of the simulation (residual velocity and residual mass of the projectile) are used as input and output data to train and validate a neural network. In addition, the number of training cases needed to arrive at a given predictive error is studied. The results are satisfactory, this alternative providing a highly recommended option for armour design tasks, due to its simplicity of handling, low computational cost and efficiency.     

一种模糊神经网络的结构和参数的确定方法*

结合FGP(Fuzzy Grid Partition,模糊网格划分)和FNN(Fuzzy Neural Network,神经网络)提出一种有效确定模糊神经网络模型中结构及参数的方法.该方法首先从样本数据中采用模糊网格划分确定出最佳规则数,从而可确定神经网络的结构;然后采用BP算法对神经网络进行调节,从而确定出模糊神经网络的参数.采用这种方法构建我国经济增长的模糊模型.研究表明这种方法构建的模糊神经网络具有更高的精度.     

Prediction and optimization of depth of penetration for stainless steel gas tungsten arc welded plates using artificial neural networks and simulated annealing algorithm

The quality of a weld joint is highly influenced by depth of penetration. Hence, accurate prediction and maximization of depth of penetration is highly essential to ensure a good-quality joint. This paper highlights the development of neural network model for predicting depth of penetration and optimizing the process parameters for maximizing depth of penetration using simulated annealing algorithm. The process parameters chosen for the study are welding current, welding speed, gas flow rate and welding gun angle. The chosen output parameter was depth of penetration. The experiments were conducted based on design of experiments using fractional factorial with 125 runs. Using the experimental data, feed-forward backpropagation neural network model was developed and trained using Levenberg–Marquardt algorithm. It was found that ANN model based on network 4-15-1 predicted depth of penetration more accurately. A mathematical model was also developed correlating the process parameters with depth of penetration for doing optimization. A source code was developed in MATLAB to do the optimization. The optimized process parameters gave a value of 3.778 mm for depth of penetration.     

GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation

ContextIn software industry, project managers usually rely on their previous experience to estimate the number men/hours required for each software project. The accuracy of such estimates is a key factor for the efficient application of human resources. Machine learning techniques such as radial basis function (RBF) neural networks, multi-layer perceptron (MLP) neural networks, support vector regression (SVR), bagging predictors and regression-based trees have recently been applied for estimating software development effort. Some works have demonstrated that the level of accuracy in software effort estimates strongly depends on the values of the parameters of these methods. In addition, it has been shown that the selection of the input features may also have an important influence on estimation accuracy.ObjectiveThis paper proposes and investigates the use of a genetic algorithm method for simultaneously (1) select an optimal input feature subset and (2) optimize the parameters of machine learning methods, aiming at a higher accuracy level for the software effort estimates.MethodSimulations are carried out using six benchmark data sets of software projects, namely, Desharnais, NASA, COCOMO, Albrecht, Kemerer and Koten and Gray. The results are compared to those obtained by methods proposed in the literature using neural networks, support vector machines, multiple additive regression trees, bagging, and Bayesian statistical models.ResultsIn all data sets, the simulations have shown that the proposed GA-based method was able to improve the performance of the machine learning methods. The simulations have also demonstrated that the proposed method outperforms some recent methods reported in the recent literature for software effort estimation. Furthermore, the use of GA for feature selection considerably reduced the number of input features for five of the data sets used in our analysis.ConclusionsThe combination of input features selection and parameters optimization of machine learning methods improves the accuracy of software development effort. In addition, this reduces model complexity, which may help understanding the relevance of each input feature. Therefore, some input parameters can be ignored without loss of accuracy in the estimations.     

Taking on the curse of dimensionality in joint distributions usingneural networks

The curse of dimensionality is severe when modeling high-dimensional discrete data: the number of possible combinations of the variables explodes exponentially. We propose an architecture for modeling high-dimensional data that requires resources (parameters and computations) that grow at most as the square of the number of variables, using a multilayer neural network to represent the joint distribution of the variables as the product of conditional distributions. The neural network can be interpreted as a graphical model without hidden random variables, but in which the conditional distributions are tied through the hidden units. The connectivity of the neural network can be pruned by using dependency tests between the variables (thus reducing significantly the number of parameters). Experiments on modeling the distribution of several discrete data sets show statistically significant improvements over other methods such as naive Bayes and comparable Bayesian networks and show that significant improvements can be obtained by pruning the network.     

Performance modeling of load-balancing algorithms using neural networks

The paper presents a new approach that uses neural networks to predict the performance of a number of dynamic decentralized load-balancing strategies. A distributed multicomputer system using distributed load-balancing strategies is represented by a unified analytical queuing model. A large simulation data set is used to train a neural network using the back-propagation learning algorithm based on gradient descent The performance model using the predicted data from the neural network produces the average response time of various load balancing algorithms under various system parameters. The validation and comparison with simulation data show that the neural network is very effective in predicting the performance of dynamic load-balancing algorithms. Our work leads to interesting techniques for designing load balancing schemes (for large distributed systems) that are computationally very expensive to simulate. One of the important findings is that performance is affected least by the number of nodes, and most by the number of links at each node in a large distributed system.     

Combining seasonal time series ARIMA method and neural networks with genetic algorithms for predicting the production value of the mechanical industry in Taiwan

Supplying industrial firms with an accurate method of forecasting the production value of the mechanical industry to facilitate decision makers in precise planning is highly desirable. Numerous methods, including the autoregressive integrated-moving average (ARIMA) model and artificial neural networks can make accurate forecasts based on historical data. The seasonal ARIMA (SARIMA) model and artificial neural networks can also handle data involving trends and seasonality. Although neural networks can make predictions, deciding the most appropriate input data, network structure and learning parameters are difficult. Therefore, this article presents a hybrid forecasting method that combines the SARIMA model and neural networks with genetic algorithms. Analytical results generated by the SARIMA model are inputted as the input data of a neural network. Subsequently, the number of neurons in the hidden layer and the number of learning parameters of the neural network architecture are globally optimized using genetic algorithms. This model is subsequently adopted to forecast seasonal time series data of the production value of the mechanical industry in Taiwan. The results presented here provide a valuable reference for decision makers in industry.     

Artificial neural network and wavelet neural network approaches for modelling of a solar air heater

This paper reports on a modelling study of new solar air heater (SAH) system by using artificial neural network (ANN) and wavelet neural network (WNN) models. In this study, a device for inserting an absorbing plate made of aluminium cans into the double-pass channel in a flat-plate SAH. A SAH system is a multi-variable system that is hard to model by conventional methods. As regards the ANN and WNN methods, it has a superior capability for generalization, and this capability is independent on the dimensionality of the input data’s. In this study, an ANN and WNN based methods were intended to adopt SAH system for efficient modelling. To evaluate prediction capabilities of different types of neural network models (ANN and WNN), their best architecture and effective training parameters should be found. The performance of the proposed methodology was evaluated by using several statistical validation parameters. Comparison between predicted and experimental results indicates that the proposed WNN model can be used for estimating the some parameters of SAHs with reasonable accuracy.