A note on phase transitions and computational pitfalls of learning from sequences

An ever greater range of applications call for learning from sequences. Grammar induction is one prominent tool for sequence learning, it is therefore important to know its properties and limits. This paper presents a new type of analysis for inductive learning. A few years ago, the discovery of a phase transition phenomenon in inductive logic programming proved that fundamental characteristics of the learning problems may affect the very possibility of learning under very general conditions. We show that, in the case of grammatical inference, while there is no phase transition when considering the whole hypothesis space, there is a much more severe “gap” phenomenon affecting the effective search space of standard grammatical induction algorithms for deterministic finite automata (DFA). Focusing on standard search heuristics, we show that they overcome this difficulty to some extent, but that they are subject to overgeneralization. The paper last suggests some directions to alleviate this problem.

A computational cognition model of perception,memory, and judgment

The mechanism of human cognition and its computability provide an important theoretical foun- dation to intelligent computation of visual media. This paper focuses on the intelligent processing of massive data of visual media and its corresponding processes of perception, memory, and judgment in cognition. In particular, both the human cognitive mechanism and cognitive computability of visual media are investigated in this paper at the following three levels： neurophysiology, cognitive psychology, and computational modeling. A computational cognition model of Perception, Memory, and Judgment （PMJ model for short） is proposed, which consists of three stages and three pathways by integrating the cognitive mechanism and computability aspects in a unified framework. Finally, this paper illustrates the applications of the proposed PMJ model in five visual media research areas. As demonstrated by these applications, the PMJ model sheds some light on the intelligent processing of visual media, and it would be innovative for researchers to apply human cognitive mechanism to computer science.     

The Supervised Learning Gaussian Mixture Model

The traditional Gaussian Mixture Model(GMM)for pattern recognition is an unsupervised learning method.The parameters in the model are derived only by the training samples in one class without taking into account the effect of sample distributions of other classes,hence,its recognition accuracy is not ideal sometimes.This paper introduces an approach for estimating the parameters in GMM in a supervising way.The Supervised Learning Gaussian Mixture Model(SLGMM)improves the recognition accuracy of the GMM.An experimental example has shown its effectiveness.The experimental results have shown that the recognition accuracy derived by the approach is higher than those obtained by the Vector Quantization(VQ)approach,the Radial Basis Function (RBF) network model,the Learning Vector Quantization (LVQ) approach and the GMM.In addition,the training time of the approach is less than that of Multilayer Perceptrom(MLP).     

Top-Down modulation of neural responses in visual perception: a computational exploration

Visual perception is typically performed in the context of a task or goal. Nonetheless, visual processing has traditionally been conceptualized in terms of a fixed, task-independent hierarchy of feature detectors. We explore the computational implications of allowing early visual processing to be task modulated. Using artificial neural networks, we show that significant improvements in task accuracy can be obtained by allowing the weights to be modulated by task. The primary benefits are obtained under resource-limited processing. A relatively modest task-based modulation of weights and activities can lead to a large performance boost, suggesting an efficient means of increasing effective cortical capacity.     

Evolution of neural networks for classification and regression

Although Artificial Neural Networks (ANNs) are importantdata mining techniques, the search for the optimal ANN is a challenging task: the ANN should learn the input–output mapping without overfitting the data and training algorithms may get trapped in local minima. The use of Evolutionary Computation (EC) is a promising alternative for ANN optimization. This work presents two hybrid EC/ANN algorithms: the first evolves neural topologies while the latter performs simultaneous optimization of architectures and weights. Sixteen real-world tasks were used to test these strategies. Competitive results were achieved when compared with a heuristic model selection and other Data Mining algorithms.     

Stereo-disparity perception for a monochromatic flat slope based on a neural network dynamic model

Stereo-matching is one of the most active research topics in computer vision. In this article, the stereo-correspondence problem for a stereo-image pair on a monochromatic surface is considered. Even if some hints exist, it is not easy to reconstruct the correct 3-D scene from two images because it is an ill-posed problem. We have modified our previous competitive and cooperative neural network model so that we can efficiently perceive a monochromatic surface which is enclosed by two vertical stripes. The modification consists of two factors: (1) combining the parameterized multiple inputs (similarities); (2) extending the cooperative terms of the neural network equation. The effect of the proposed model is examined by experiments with both synthetic and real stereo-image pairs. For the real images, a segmentation method is proposed to deal with the similarity maps. This work was presented, in part, at the Seventh International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

Improved competitive learning neural networks for network intrusion and fraud detection

In this research, we propose two new clustering algorithms, the improved competitive learning network (ICLN) and the supervised improved competitive learning network (SICLN), for fraud detection and network intrusion detection. The ICLN is an unsupervised clustering algorithm, which applies new rules to the standard competitive learning neural network (SCLN). The network neurons in the ICLN are trained to represent the center of the data by a new reward-punishment update rule. This new update rule overcomes the instability of the SCLN. The SICLN is a supervised version of the ICLN. In the SICLN, the new supervised update rule uses the data labels to guide the training process to achieve a better clustering result. The SICLN can be applied to both labeled and unlabeled data and is highly tolerant to missing or delay labels. Furthermore, the SICLN is capable to reconstruct itself, thus is completely independent from the initial number of clusters.To assess the proposed algorithms, we have performed experimental comparisons on both research data and real-world data in fraud detection and network intrusion detection. The results demonstrate that both the ICLN and the SICLN achieve high performance, and the SICLN outperforms traditional unsupervised clustering algorithms.     

A neural network model of the olfactory system of mice: simulated the tendency of attention behavior

The demands for odor processing apparatuses have been increasing in fragrance or food industries. However, odors are extremely high dimensional information composed a combination from tens thousands of different odorant molecules, and thus requires vast amounts of computation. Therefore, it is considered learning from a living nose would be an efficient approach. From the odor discrimination experiments, it was found that mice have a feature extraction ability called Attention by which they could focus on the important odorants for odor discrimination. In this paper we propose a neural network model approximated to actual number of neurons and the structure of olfactory system. Simulation experiments of the proposed model were implemented based on the odor discrimination experiments on the living mice. From the simulation results of the model, we confirmed not only the proposed model had ability of Attention, but also the tendency of Attention was consistent with the living mice. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Neural network control of nonlinear dynamic systems using hybrid algorithm

In this paper, a hybrid method is proposed to control a nonlinear dynamic system using feedforward neural network. This learning procedure uses different learning algorithm separately. The weights connecting the input and hidden layers are firstly adjusted by a self organized learning procedure, whereas the weights between hidden and output layers are trained by supervised learning algorithm, such as a gradient descent method. A comparison with backpropagation (BP) shows that the new algorithm can considerably reduce network training time.     

A deep learning approach to the classification of 3D CAD models简

Model classification is essential to the management and reuse of 3D CAD models. Manual model classification is laborious and error prone. At the same time, the automatic classification methods are scarce due to the intrinsic complexity of 3D CAD models. In this paper, we propose an automatic 3D CAD model classification approach based on deep neural networks. According to prior knowledge of the CAD domain, features are selected and extracted from 3D CAD models first, and then pre-processed as high dimensional input vectors for category recognition. By analogy with the thinking process of engineers, a deep neural network classifier for 3D CAD models is constructed with the aid of deep learning techniques. To obtain an optimal solution, multiple strategies are appropriately chosen and applied in the training phase, which makes our classifier achieve better per-formance. We demonstrate the efficiency and effectiveness of our approach through experiments on 3D CAD model datasets.     

A neural network model of the olfactory system of mice: computer simulation of the attention behavior of mice for some components in an odor

Recently, it was observed that mice could identify an odor by paying attention to only a few of its components. Further, it has been reported that each individual is attracted to different components of an odor. This behavior is referred to as “attention”; however, its mechanism has yet to be completely elucidated. In this paper, we first propose a novel artificial neural network model based on the biological structure of an olfactory system. Then a series of computer simulations of odorant discrimination are performed to evaluate the attention ability of the proposed model. Finally, we changed the connective weights between the neurons to simulate individual differences. The simulation results indicate that the inhibitory connections from the piriform cortex to the olfactory bulb may contribute to the individual differences observed in the behavioral experiment. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

Using the BSP cost model to optimise parallel neural network training

We derive cost formulae for three different parallelisation techniques for training both supervised and unsupervised networks. These formulae are parameterised by properties of the target computer architecture. It is therefore possible to decide both which technique is best for a given parallel computer, and which parallel computer best suits a given technique. One technique, exemplar parallelism, is far superior to almost all parallel computer architectures. Formulae also take into account optimal batch learning as the overall training approach. Cost predictions are made for several of today's popular parallel computers.     

A computational model for static data flow machines

Computer architects have been constantly looking for new approaches to design high-performance machines. Data flow and VLSI offer two mutually supportive approaches towards a promising design for future super-computers. When very high speed computations are needed, data flow machines may be relied upon as an adequate solution in which extremely parallel processing is achieved.

This paper presents a formal analysis for data flow machines. Moreover, the following three machines are considered: (1) MIT static data flow machine; (2) TI's DDP static data flow machine; (3) LAU data flow machine.

These machines are investigated by making use of a reference model. The contributions of this paper include: (1) Developing a Data Flow Random Access Machine model (DFRAM), for first time, to serve as a formal modeling tool. Also, by making use of this model one can calculate the time cost of various static data machines, as well as the performance of these machines. (2) Constructing a practical Data Flow Simulator (DFS) on the basis of the DFRAM model. Such DFS is modular and portable and can be implemented with less sophistication. The DFS is used not only to study the performance of the underlying data flow machines but also to verify the DFRAM model.     

Associative learning method in a hypercolumn model

We propose an associatively learnable hypercolumn model (AHCM). A hyper-column model is a self-organized, competitive, and hierarchical multilayer neural network. It is derived from the neocognitron by replacing each S cell and C cell with a two-layer hierarchical self-organizing map. The HCM can recognize images with variant object size, position, orientation and spatial resolution. However, feature maps may integrate some features extracted in the lower layer even if the features are extracted from input data which belong to different categories. The learning algorithm of the HCM causes this problem because it is an unsupervised learning used by a self-organizing map. An associative learning method is therefore introduced, which is derived from the HCM by appending associative signals and associative weights to traditional input data and connection weights, respectively. The AHCM was applied to hand-shape recognition. We found that the AHCM could generate an appropriate feature map and higher recognition accuracy compared with the HCM. This work was presented in part at the 11th International Symposium on Artificial Life and Robotics, Oita, Japan, January 23–25, 2006     

Artificial wavelet neuro-fuzzy model based on parallel wavelet network and neural network

From the well-known advantages and valuable features of wavelets when used in neural network, two type of networks (i.e., SWNN and MWNN) have been proposed. These networks are single hidden layer network. Each neuron in the hidden layer is comprised of wavelet and sigmoidal activation functions. First model is derived from adding the outputs of wavelet and sigmoidal activation functions, while in the second model outputs of wavelet and sigmoidal activation function are multiplied together. Using these proposed networks in consequent part of the neuro-fuzzy model, which result summation wavelet neuro-fuzzy and multiplication wavelet neuro-fuzzy models, are also proposed. Different types of wavelet function are tested with proposed networks and fuzzy models on four different types of examples. Convergence of the learning process is also guaranteed by performing stability analysis using Lyapunov function.     

基于空间维度循环感知网络的密集人群计数模型

考虑目前对具有透视畸变的高密度人群图像进行特征提取的局限性,提出了一种融合全局特征感知网络(GFPNet)和局部关联性特征感知网络(LAFPNet)的人群计数模型LMCNN.GFPNet是LMCNN的主干网络,将其输出的特征图进一步序列化并作为LAFPNet的输入,再利用循环神经网络(RNN)在时序维度上对局部关联性特...     

A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning

In cost-sensitive learning, misclassification costs can vary for different classes. This paper investigates an approach reducing a multi-class cost-sensitive learning to a standard classification task based on the data space expansion technique developed by Abe et al., which coincides with Elkan's reduction with respect to binary classification tasks. Using this proposed reduction approach, a cost-sensitive learning problem can be solved by considering a standard 0/1 loss classification problem on a new distribution determined by the cost matrix. We also propose a new weighting mechanism to solve the reduced standard classification problem, based on a theorem stating that the empirical loss on independently identically distributed samples from the new distribution is essentially the same as the loss on the expanded weighted training set. Experimental results on several synthetic and benchmark datasets show that our weighting approach is more effective than existing representative approaches for cost-sensitive learning.     

基于改进的BP网络模型的分类器的设计与实现

用改进的ＢＰ网络模型作了分类器。改进的模型拓扑结构最简，学习速率快，分类准确率高。     

A fuzzy logic-based computational recognition-primed decision model

The recognition-primed decision (RPD) model is a primary naturalistic decision-making approach which seeks to explicitly recognize how human decision makers handle complex tasks and environment based on their experience. Motivated by the need for quantitative computer modeling and simulation of human decision processes in various application domains, including medicine, we have developed a general-purpose computational fuzzy RPD model that utilizes fuzzy sets, fuzzy rules, and fuzzy reasoning to represent, interpret, and compute imprecise and subjective information in every aspect of the model. Experiences acquired by solicitation with experts are stored in experience knowledge bases. New local and global similarity measures have been developed to identify the experience that is most applicable to the current situation in a specific decision-making context. Furthermore, an action evaluation strategy has been developed to select the workable course of action. The proposed fuzzy RPD model has been preliminarily validated by using it to calculate the extent of causality between a drug (Cisapride, withdrawn by the FDA from the market in 2000) and some of its adverse effects for 100 hypothetical patients. The simulated patients were created based on the profiles of over 1000 actual patients treated with the drug at our medical center before its withdrawal. The model validity was demonstrated by comparing the decisions made by the proposed model and those by two independent internists. The levels of agreement were established by the weighted Kappa statistic and the results suggested good to excellent agreement.