When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Parallel path consistency

Filtering algorithms are well accepted as a means of speeding up the solution of the consistent labeling problem (CLP). Despite the fact that path consistency does a better job of filtering than arc consistency, AC is still the preferred technique because it has a much lower time complexity. We are implementing parallel path consistency algorithms on multiprocessors and comparing their performance to the best sequential and parallel arc consistency algorithms.^(1,2) (See also work by Kerethoet al. ⁽³⁾ and Kasif⁽⁴⁾) Preliminary work has shown linear performance increases for parallelized path consistency and also shown that in many cases performance is significantly better than the theoretical worst case. These two results lead us to believe that parallel path consistency may be a superior filtering technique. Finally, we have implemented path consistency as an outer product computation and have obtained good results (e.g., linear speedup on a 64K-node Connection Machine 2).     

Disorder effects on the strain response of model polymer networks

Molecular dynamics simulations are used to investigate polymer networks made by either end-linking or randomly crosslinking a melt of linear precursor chains. The resulting network structures are very different, since end-linking leads to nearly ideal monodisperse networks, while random crosslinking leads to polydisperse networks, characterized by an exponential strand length distribution. Networks with average strand length 20 and 100 were generated. These networks were used to study the effects of disorder in the network connectivity on observables averaged either over the entire network or selected sub-structures. Heterogeneities in the randomly crosslinked networks cause significant differences in the localization of monomers, however, neither the localization of crosslinks nor the microscopic strain response are significantly affected. Compared to end-linked networks, randomly crosslinked networks have a slightly increased tube diameter, and as a result a slightly decreased shear modulus, but otherwise identical stress-strain behavior. For the investigated systems, we conclude that the microscopic strain response, tube diameter, and stress-strain relation are all insensitive to the heterogeneities due to the linking process by which the network were made.     

The five elements and chinese-american mortality.

D. P. Phillips, T. E. Ruth, and L. M. Wagner (1993) reported that 1969-1990 California mortality data show that Chinese Americans are particularly vulnerable to diseases that Chinese astrology and traditional Chinese medicine associate with their birth years. For example, because fire is associated with the heart, a Chinese person born in a fire year (such as 1937) is more likely to die of heart disease than is a Chinese person born in a nonfire year. However, many diseases were excluded from this study, some diseases that were included have ambiguous links to birth years, and the statistical tests were indirect. A more complete statistical analysis and independent California mortality data for the years 1960-1968 and 1991-2002 did not replicate the original results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Employing optimized combinations of one-class classifiers for automated currency validation

Automated currency validation requires a decision to be made regarding the authenticity of a banknote presented to the validation system. This decision often has to be made with little or no information regarding the characteristics of possible counterfeits as is the case for issues of new currency. A method for automated currency validation is presented which segments the whole banknote into different regions, builds individual classifiers on each region and then combines a small subset of the region specific classifiers to provide an overall decision. The segmentation and combination of region specific classifiers to provide optimized false positive and false negative rates is achieved by employing a genetic algorithm. Experiments based on high value notes of Sterling currency were carried out to assess the effectiveness of the proposed solution.     

Role of Ponded Turbidity Currents in Reservoir Trap Efficiency

The capacity to store water in a reservoir declines as it traps sediment. A river entering a reservoir forms a prograding delta. Coarse sediment (e.g., sand) deposits in the fluvial topset and avalanching foreset of the delta, and is typically trapped with an efficiency near 100%. The trap efficiency of fine sediment (e.g., mud), on the other hand, may be below 100%, because some of this sediment may pass out of the reservoir without settling out. Here, a model of trap efficiency of mud is developed in terms of the mechanics of a turbidity current that plunges on the foreset. The dam causes a sustained turbidity current to reflect and form a muddy pond bounded upstream by a hydraulic jump. If the interface of this muddy pond rises above any vent or overflow point at the dam, the trap efficiency of mud drops below 100%. A model of the coevolution of topset, foreset, and bottomset in a reservoir that captures the dynamics of the internal muddy pond is presented. Numerical implementation, comparison against an experiment, and application to a field-scale case provide the basis for a physical understanding of the processes that determine reservoir trap efficiency.     

The Lambda Library: unnamed functions in C++

The Lambda Library (LL) adds a form of lambda functions to C++, which are common in functional programming languages. The LL is implemented as a template library using standard C++; thus no language extensions or preprocessing is required. The LL consists of a rich set of tools for defining unnamed functions. In particular these unnamed functions work seamlessly with the generic algorithms in the C++ Standard Library. The LL offers significant improvements, in terms of generality and ease of use, compared to the current tools in the C++ Standard Library. Copyright © 2003 John Wiley & Sons, Ltd.     

Adolescent abortion: Psychological perspectives on public policy.

The Supreme Court has relied on psychological assumptions in adolescent abortion cases, but it has failed to consider relevant empirical research. The work of the Interdivisional Committee on Adolescent Abortion to fill in this gap provides a model for organized psychology's integration, dissemination, and application of psychological knowledge to promote the public interest. Such efforts should be replicated by state associations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Quantitative evaluation of pancreatic enhancement during dual-phase helical CT

PURPOSE: To determine the improvement in pancreatic enhancement at helical computed tomography (CT) performed with an early delay after administration of contrast material compared with that performed with a standard delay. MATERIALS AND METHODS: Dual-phase helical CT of the abdomen was performed in 120 patients with a 150-mL bolus of contrast material infused at 5 mL/sec. Early and standard delayed scanning was performed beginning at 20 seconds and 49-71 seconds, respectively. Regions of interest were measured in the head, body, and tail of the pancreas in 92 patients. The difference in enhancement between early and standard delayed scanning was calculated. RESULTS: Mean pancreatic enhancement was 82 HU +/- 3 (standard error) with an early delay, whereas enhancement on standard delay scans was 62 HU +/- 2 (P < .001). An improvement in enhancement greater than 10 HU was attained in 66 of 92 cases (72%). CONCLUSION: Pancreatic enhancement at helical CT with an early delay after contrast material administration is often significantly greater than the enhancement seen with a standard delay when a monophasic, rapidly infused bolus of contrast material is used.