Multiagent learning using a variable learning rate

Learning to act in a multiagent environment is a difficult problem since the normal definition of an optimal policy no longer applies. The optimal policy at any moment depends on the policies of the other agents. This creates a situation of learning a moving target. Previous learning algorithms have one of two shortcomings depending on their approach. They either converge to a policy that may not be optimal against the specific opponents' policies, or they may not converge at all. In this article we examine this learning problem in the framework of stochastic games. We look at a number of previous learning algorithms showing how they fail at one of the above criteria. We then contribute a new reinforcement learning technique using a variable learning rate to overcome these shortcomings. Specifically, we introduce the WoLF principle, “Win or Learn Fast”, for varying the learning rate. We examine this technique theoretically, proving convergence in self-play on a restricted class of iterated matrix games. We also present empirical results on a variety of more general stochastic games, in situations of self-play and otherwise, demonstrating the wide applicability of this method.     

基于约束优化的联想记忆模型学习算法

本文提出了一种对称互连神经元网络的学习策略,利用全局约束优化方法确定连接权。优化过程采用了梯度下降技术。这种学习算法可以保证训练样本成为系统的稳定吸引子,并且具有优化意义上的最大吸引域。本文讨论了网络的存储容量,训练样本的渐近稳定性和吸引域大小。计算机实验结果说明了学习算法的优越性。     

Hierarchical intelligent control with flexible AC transmission systems application

A novel hierarchical intelligent controller configuration is proposed using an artificial neural network as a control-mode classifier in the supervisory level and a set of pre-designed controllers in the lower level. Controller outputs are modified nonlinearly by the classifying signals in a structure resembling one artificial neuron with adaptively changed weights. The lower-level local controllers are implemented using neural networks. An illustrative example of this approach is based on the transient stabilization of a single-machine infinite-bus system studied in Flexible AC Transmission Systems (FACTS) research.     

Knowledge extraction for production management

A procedure and underlying algorithm for extracting knowledge from production and inventory databases to support engineering management activities is described. The process searches for, detects and isolates behaviour patterns inherent in the data. It relates these patterns to production irregularities, suggests connections with specific causes and helps propose possible corrective or preventive actions. The approach is based on a four-phase procedure: (1) the decision-maker focuses on the subject or difficulty at issue, represented by a target concept; (2) the KEDB algorithm, based on a machine learning approach, processes the relevant database and provides knowledge characterizing and classifying the target concept; (3) the output is interpreted in Pareto fashion as a series of possible circumstances explaining the target concept behaviour; and (4) based on these causes, the decision-maker decides on possible corrective actions to improve the situation, or preventive actions to forestall unfavourable conditions. A case study based on an actual quality control database is detailed.     

Distinguished Scientific Awards for an Early Career Contribution to Psychology.

The Early Career Awards recognize the large number of excellent young psychologists. Recipients of this award may not have held a PhD for more than nine years. The areas considered in 1991 were animal learning and behavior, developmental, and personality. The winners for 1991 were Kent C. Berridge, Susan A. Gelman, and Randy J. Larsen. For each winner a citation, biography, and selected bibliography of their works are presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Neural Network Feedback Control: Work at UTA’s Automation and Robotics Research Institute

This is an outline of research in neural networks for feedback control done since the mid 1990s at the Automation and Robotics Research Institute (ARRI) of The University of Texas at Arlington (UTA). It shows how the developments of Intelligent Control Systems based on neural networks have followed three main generations. This statement provides a short, broad-brush perspective on the development of intelligent neural feedback controllers.     

Feedback Specificity, Learning Opportunities, and Learning.

Although increasing feedback specificity is generally beneficial for immediate performance, it can undermine certain aspects of the learning needed for later, more independent performance. The results of the present transfer experiment demonstrate that the effects of increasing feedback specificity on learning depended on what was to be learned, and these effects were partially mediated through the opportunities to learn how to respond to different task conditions during practice. More specific feedback was beneficial for learning how to respond to good performance and detrimental for learning how to respond to poor performance. The former relationship was partially mediated by feedback specificity's effect on learning opportunities during practice. The results have implications for designing feedback interventions and training to maximize the learning of various aspects of a task. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Genetic Dissociation of Learning and Recall in Caenorhabditis elegans.

A learning event can be dissociated into 3 components: acquisition, storage, and recall. When the laboratory wild-type strain of Caenorhabditis elegans (N2 strain) is exposed to benzaldehyde in the absence of food, the worms display a reduction of their attractive response to this volatile odorant. This results from the association between benzaldehyde and a nutrient-deficient environment. Another wild-type isolate, the CB4856 strain, fails to display this decreased response to benzaldehyde after exposure to benzaldehyde in the absence of food. However, like the N2 strain, when tested to isoamyl alcohol after benzaldehyde conditioning, the CB4856 strain displays a decreased isoamyl alcohol response. Therefore, the CB4856 strain does not have an acquisition deficit, but it suffers from a recall deficit specific to benzaldehyde. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pure perceptual-based sequence learning.

Learning a sequence of target locations when the sequence is uncorrelated with a sequence of responses and target location is not the response dimension (pure perceptual-based sequence learning) was examined. Using probabilistic sequences of target locations, the author shows that such learning can be implicit, is unaffected by distance between target locations, and is mostly limited to first-order transition probabilities. Moreover, the mechanism underlying learning affords processing of information at anticipated target locations and appears to be attention based. Implications for hypotheses of implicit sequence learning are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Dynamic Dictionary for Priced Information with Application

In this paper we design a dynamic dictionary for the priced information model initiated by Charikar et al. Assume that a set S consisting of n elements is given such that each element has an associated price, a positive real number. The cost of performing an operation on elements of S is a function of their prices. The cost of an algorithm is the sum of the costs of all operations it performs. The objective is to design algorithms which incur low cost. In this model we propose a dynamic dictionary, supporting search, insert and delete, for keys drawn from a linearly ordered set. As an application we show that the dictionary can be used in computing the trapezoidal map of a set of line segments, a fundamental problem in computational geometry.