Supervised extended ART: A fast neural network classifier trained by combining supervised and unsupervised learning

A neural network classifier, called supervised extended ART (SEART), that incorporates a supervised mechanism into the extended unsupervised ART is presented here. It uses a learning theory called Nested Generalized Exemplar (NGE) theory. In any time, the training instances may or may not have desired outputs, that is, this model can handle supervised learning and unsupervised learning simultaneously. The unsupervised component finds the cluster relations of instances, and the supervised component learns the desired associations between clusters and classes. In addition, this model has the ability of incremental learning. It works equally well when instances in a cluster belong to different classes. Also, multi-category and nonconvex classifications can be dealt with. Besides, the experimental results are very encouraging.     

Clustering and group selection of multiple criteria alternatives with application to space-based networks

In many real-world problems, the range of consequences of different alternatives are considerably different. In addition, sometimes, selection of a group of alternatives (instead of only one best alternative) is necessary. Traditional decision making approaches treat the set of alternatives with the same method of analysis and selection. In this paper, we propose clustering alternatives into different groups so that different methods of analysis, selection, and implementation for each group can be applied. As an example, consider the selection of a group of functions (or tasks) to be processed by a group of processors. The set of tasks can be grouped according to their similar criteria, and hence, each cluster of tasks to be processed by a processor. The selection of the best alternative for each clustered group can be performed using existing methods; however, the process of selecting groups is different than the process of selecting alternatives within a group. We develop theories and procedures for clustering discrete multiple criteria alternatives. We also demonstrate how the set of alternatives is clustered into mutually exclusive groups based on 1) similar features among alternatives; 2) ideal (or most representative) alternatives given by the decision maker; and 3) other preferential information of the decision maker. The clustering of multiple criteria alternatives also has the following advantages. 1) It decreases the set of alternatives to be considered by the decision maker (for example, different decision makers are assigned to different groups of alternatives). 2) It decreases the number of criteria. 3) It may provide a different approach for analyzing multiple decision makers problems. Each decision maker may cluster alternatives differently, and hence, clustering of alternatives may provide a basis for negotiation. The developed approach is applicable for solving a class of telecommunication networks problems where a set of objects (such as routers, processors, or intelligent autonomous vehicles) are to be clustered into similar groups. Objects are clustered based on several criteria and the decision maker's preferences.     

Neurocontroller design via supervised and unsupervised learning

In this paper we study the role of supervised and unsupervised neural learning schemes in the adaptive control of nonlinear dynamic systems. We suggest and demonstrate that the teacher's knowledge in the supervised learning mode includes a-priori plant sturctural knowledge which may be employed in the design of exploratory schedules during learning that results in an unsupervised learning scheme. We further demonstrate that neurocontrollers may realize both linear and nonlinear control laws that are given explicitly in an automated teacher or implicitly through a human operator and that their robustness may be superior to that of a model based controller. Examples of both learning schemes are provided in the adaptive control of robot manipulators and a cart-pole system.     

A design and analysis of objective function-based unsupervised neural networks for fuzzy clustering

Fuzzy clustering has played an important role in solving many problems. In this paper, we design an unsupervised neural network model based on a fuzzy objective function, called OFUNN. The learning rule for the OFUNN model is a result of the formal derivation by the gradient descent method of a fuzzy objective function. The performance of the cluster analysis algorithm is often evaluated by counting the number of crisp clustering errors. However, the number of clustering errors alone is not a reliable and consistent measure for the performance of clustering, especially in the case of input data with fuzzy boundaries. We introduce two measures to evaluate the performance of the fuzzy clustering algorithm. The clustering results on three data sets, Iris data and two artificial data sets, are analyzed using the proposed measures. They show that OFUNN is very competitive in terms of speed and accuracy compared to the fuzzy c-means algorithm.     

Six-Sigma project selection using national quality award criteria and Delphi fuzzy multiple criteria decision-making method

Six-Sigma is a tactical tool of significant value in achieving operational excellence. The project selection decision, under a resources constraint, is the early stage of implementation for a Six-Sigma intervention. The project selection decision is challenging due to its fuzzy group decision-making aspect inherent to the problem. The present study proposes to adopt national quality award criteria as the Six-Sigma project selection criteria, and proposes a hierarchical criteria evaluation process. The strategic criteria are evaluated by the management team using a Delphi fuzzy multiple criteria decision-making method. Then, the tactical sub-criteria which contain additional operational issues are evaluated by the Six-Sigma Champion. The proposed methodology is successfully applied in solving the project selection problem deriving from a component manufacturer. The empirical outcomes are promising. Moreover, the results show that the higher a project’s priority is, the greater the financial gains will be on completion of the project. Accordingly, the proposed methodology can prioritize the financial gain – which is the key performance indicator for a Six-Sigma project. Additionally, the quality status of the case company has been significantly improved through implementation of the Six-Sigma project. The systematic evaluation process also influences employees to adopt an analytical operations philosophy. Moreover, the commercial objectives of the company are brought into focus by the proposed methodology.     

Classification and weakly supervised pain localization using multiple segment representation

Automatic pain recognition from videos is a vital clinical application and, owing to its spontaneous nature, poses interesting challenges to automatic facial expression recognition (AFER) research. Previous pain vs no-pain systems have highlighted two major challenges: (1) ground truth is provided for the sequence, but the presence or absence of the target expression for a given frame is unknown, and (2) the time point and the duration of the pain expression event(s) in each video are unknown. To address these issues we propose a novel framework (referred to as MS-MIL) where each sequence is represented as a bag containing multiple segments, and multiple instance learning (MIL) is employed to handle this weakly labeled data in the form of sequence level ground-truth. These segments are generated via multiple clustering of a sequence or running a multi-scale temporal scanning window, and are represented using a state-of-the-art Bag of Words (BoW) representation. This work extends the idea of detecting facial expressions through ‘concept frames’ to ‘concept segments’ and argues through extensive experiments that algorithms such as MIL are needed to reap the benefits of such representation.     

Model-free visualization of suspicious lesions in breast MRI based on supervised and unsupervised learning

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has become an important tool in breast cancer diagnosis, but evaluation of multitemporal 3D image data holds new challenges for human observers. To aid the image analysis process, we apply supervised and unsupervised pattern recognition techniques for computing enhanced visualizations of suspicious lesions in breast MRI data. These techniques represent an important component of future sophisticated computer-aided diagnosis (CAD) systems and support the visual exploration of spatial and temporal features of DCE-MRI data stemming from patients with confirmed lesion diagnosis. By taking into account the heterogeneity of cancerous tissue, these techniques reveal signals with malignant, benign and normal kinetics. They also provide a regional subclassification of pathological breast tissue, which is the basis for pseudo-color presentations of the image data. Intelligent medical systems are expected to have substantial implications in healthcare politics by contributing to the diagnosis of indeterminate breast lesions by non-invasive imaging.     

The evaluation of air transportation network based on multiple criteria

The problem is partitioned into two components: to propose the air transportation network based on “hub and spoke” pattern as alternative network in Sumatera island (one of the islands in Indonesia), and to propose the algorithm for the aircraft selection of the suitable type of aircraft in the operation on airport pairs. The algorithm presented in this paper is obtained from heuristic, and then comparing of our proposal with the existing one is done, based on the multiple criteria.     

Learning from partially supervised data using mixture models and belief functions

This paper addresses classification problems in which the class membership of training data are only partially known. Each learning sample is assumed to consist of a feature vector x_i∈X and an imprecise and/or uncertain “soft” label m_i defined as a Dempster-Shafer basic belief assignment over the set of classes. This framework thus generalizes many kinds of learning problems including supervised, unsupervised and semi-supervised learning. Here, it is assumed that the feature vectors are generated from a mixture model. Using the generalized Bayesian theorem, an extension of Bayes’ theorem in the belief function framework, we derive a criterion generalizing the likelihood function. A variant of the expectation maximization (EM) algorithm, dedicated to the optimization of this criterion is proposed, allowing us to compute estimates of model parameters. Experimental results demonstrate the ability of this approach to exploit partial information about class labels.     

Steganalysis and payload estimation of embedding in pixel differences using neural networks

In this paper a steganalysis technique is proposed for pixel-value differencing method. This steganographic method, which is immune against conventional attacks, performs the embedding in the difference of the values of pixel pairs. Therefore, the histogram of the differences of an embedded image is different as compared with a cover image. A number of characteristics are identified in the difference histogram that show meaningful alterations when an image is embedded. Five distinct multilayer perceptrons neural networks are trained to detect different levels of embedding. Every image is fed to all networks and a voting system categorizes the image as stego or cover. The implementation results indicate 88.6% success in correct categorization of the test images that contained more than 20% embedding. Furthermore, using a neural network an estimator is presented which gives an estimate of the amount of the MPVD embedding in an image. Implementation of the estimator showed an average accuracy of 88.3% in the estimation of the amount of embedding.     

Milling force prediction using regression and neural networks

This study focuses on developing a good empirical relationship between the cutting force in an end milling operation and the cutting parameters such as speed, feed and depth-of-cut, by using both multiple regression and neural network modeling processes. A regression model was first fitted to experimentally collected data and any abnormal data points indicated by this analysis were filtered out. By repeating this process several times, a final set of filtered data was obtained and analyzed using neural networks to yield a good, final model. This study shows that analyzing milling force data using conventional regression can lead to a more accurate neural networks model for force prediction.     

A new convex objective function for the supervised learning of single-layer neural networks

This paper proposes a novel supervised learning method for single-layer feedforward neural networks. This approach uses an alternative objective function to that based on the MSE, which measures the errors before the neuron's nonlinear activation functions instead of after them. In this case, the solution can be easily obtained solving systems of linear equations, i.e., requiring much less computational power than the one associated with the regular methods. A theoretical study is included to proof the approximated equivalence between the global optimum of the objective function based on the regular MSE criterion and the one of the proposed alternative MSE function.Furthermore, it is shown that the presented method has the capability of allowing incremental and distributed learning. An exhaustive experimental study is also presented to verify the soundness and efficiency of the method. This study contains 10 classification and 16 regression problems. In addition, a comparison with other high performance learning algorithms shows that the proposed method exhibits, in average, the highest performance and low-demanding computational requirements.     

Line flow contingency selection and ranking using cascade neural network

Line flow or real-power contingency selection and ranking is performed to choose the contingencies that cause the worst overloading problems. In this paper, a cascade neural network-based approach is proposed for fast line flow contingency selection and ranking. The developed cascade neural network is a combination of a filter module and a ranking module. All the contingency cases are applied to the filter module, which is trained to classify them either in critical contingency class or in non-critical contingency class using a modified BP algorithm. The screened critical contingencies are passed to the ranking module (four-layered feed-forward artificial neural network (ANN)) for their further ranking. Effectiveness of the proposed ANN-based method is demonstrated by applying it for contingency screening and ranking at different loading conditions for IEEE 14-bus system. Once trained, the cascade neural network gives fast and accurate screening and ranking for unknown patterns and is found to be suitable for on-line applications at energy management centre.     

Integrated optimal topology design and shape optimization using neural networks

In this paper, neural network- and feature-based approaches are introduced to overcome current shortcomings in the automated integration of topology design and shape optimization. The topology optimization results are reconstructed in terms of features, which consist of attributes required for automation and integration in subsequent applications. Features are defined as cost-efficient simple shapes for manufacturing. A neural network-based image-processing technique is presented to match the arbitrarily shaped holes inside the structure with predefined features. The effectiveness of the proposed approach in integrating topology design and shape optimization is demonstrated with several experimental examples.     

典型煤岩反射光谱无监督感知方法研究

针对现有煤岩反射光谱有监督识别方法存在煤岩位置变化时识别效果差的问题,为研究基于反射光谱的煤岩自主适应性识别,提出了基于聚类距离改进型模糊C均值聚类(FCM)算法的典型煤岩反射光谱无监督感知方法。以兴隆庄煤矿气煤、泥岩、粉砂岩、泥质灰岩4种典型煤岩样品为研究对象,测定了每种试样多个背向反射角下的近红外波段的反射光谱曲线,分析了4种煤岩反射光谱最具差异性的特征波段,选取2150~2400 nm作为4种煤岩反射光谱差异性特征波段,在特征波段内,对气煤-泥岩、气煤-粉砂岩、气煤-泥质灰岩光谱组合进行煤岩反射光谱无监督识别研究。研究结果表明:4种试样表面的背向光谱反射率均呈现出随背向反射角增大而先增大后减小的整体趋势,背向反射角增大时,泥岩、粉砂岩和泥质灰岩的各吸收谷深度变化较小,只有微弱的减小,气煤的各吸收谷深度减小相对明显;采用改进型FCM(RFCM,CFCM)方法将光谱数据快速聚类,由最终聚类隶属度概率矩阵判定光谱数据类别,进而判定不同位置煤岩类别;相较于FCM,改进型FCM对各煤岩组合的识别率均大于90%,其中CFCM对各煤岩组合聚类识别迭代次数最少,总耗时均小于0.1 s,为优先选择方法,为反射光谱技术应用于煤岩界面不同位置煤岩的高效适应性判定提供了参考。     

Variable selection in regression models using nonstandard optimisation of information criteria

The question of variable selection in a regression model is a major open research topic in econometrics. Traditionally two broad classes of methods have been used. One is sequential testing and the other is information criteria. The advent of large datasets used by institutions such as central banks has exacerbated this model selection problem. A solution in the context of information criteria is provided in this paper. The solution rests on the judicious selection of a subset of models for consideration using nonstandard optimisation algorithms for information criterion minimisation. In particular, simulated annealing and genetic algorithms are considered. Both a Monte Carlo study and an empirical forecasting application to UK CPI inflation suggest that the proposed methods are worthy of further consideration.     

Nonlinear multivariable adaptive control using multiple models and neural networks

In this paper, a multivariable adaptive control approach is proposed for a class of unknown nonlinear multivariable discrete-time dynamical systems. By introducing a k-difference operator, the nonlinear terms of the system are not required to be globally bounded. The proposed adaptive control scheme is composed of a linear adaptive controller, a neural-network-based nonlinear adaptive controller and a switching mechanism. The linear controller can assure boundedness of the input and output signals, and the neural network nonlinear controller can improve performance of the system. By using the switching scheme between the linear and nonlinear controllers, it is demonstrated that improved performance and stability can be achieved simultaneously. Theory analysis and simulation results are presented to show the effectiveness of the proposed method.     

Design of laminated composite structures for optimum fiber direction and layer thickness,using optimality criteria

In this study, two optimality criteria are presented for optimum design of composite laminates using finite element method. Thickness of the layers and fiber orientation angles in each finite element are considered as the design variables. It will be shown that the optimum design of composite laminates with varying fiber orientations and layers thicknesses may be found by using these optimality criteria in an efficient way, without performing the sensitivity analysis.     

A comparison between functional networks and artificial neural networks for the prediction of fishing catches

In recent years, functional networks have emerged as an extension of artificial neural networks (ANNs). In this article, we apply both network techniques to predict the catches of the Prionace Glauca (a class of shark) and the Katsowonus Pelamis (a variety of tuna, more commonly known as the Skipjack). We have developed an application that will help reduce the search time for good fishing zones and thereby increase the fleets competitivity. Our results show that, thanks to their superior learning and generalisation capacities, functional networks are more efficient than ANNs. Our data proceeds from remote sensors. Their spectral signatures allow us to calculate products that are useful for ecological modelling. After an initial phase of digital image processing, we created a database that provides all the necessary patterns to train both network types.     

Dispatching rule selection using artificial neural networks for dynamic planning and scheduling

To schedule a job shop, the first task is to select an appropriate scheduling algorithm or rule. Because of the complexity of scheduling problems, no general algorithm sufficient for solving all scheduling problems has yet been developed. Most job-shop scheduling systems offer alternative algorithms for different situations, and experienced human schedulers are needed to select the best dispatching rule in these systems. This paper proposes a new algorithm for job-shop scheduling problems. This algorithm consists of three stages. First, computer simulation techniques are used to evaluate the efficiency of heuristic rules in different scheduling situations. Second, the simulation results are used to train a neural network in order to capture the knowledge which can be used to select the most efficient heuristic rule for each scheduling situation. Finally, the trained neural network is used as a dispatching rule selector in the real-time scheduling process. Research results have shown great potential in using a neural network to replace human schedulers in selecting an appropriate approach for real-time scheduling. This research is part of an ongoing project of developing a real-time planning and scheduling system.