When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Electronic Properties of the Layered Perovskite Ruthenates: Correlated Electron Physics Approaching the Low-Disorder Limit

We present a brief review of the physics that is now accessible in the layered perovskite ruthenates that have been the focus of our research for the past few years. The main theme of the work has been the discovery of novel ground states that have an extreme sensitivity to disorder. We discuss these developments and the lessons that they provide for the wider field of correlated electrons in solids.     

Quantitative evaluation of pancreatic enhancement during dual-phase helical CT

PURPOSE: To determine the improvement in pancreatic enhancement at helical computed tomography (CT) performed with an early delay after administration of contrast material compared with that performed with a standard delay. MATERIALS AND METHODS: Dual-phase helical CT of the abdomen was performed in 120 patients with a 150-mL bolus of contrast material infused at 5 mL/sec. Early and standard delayed scanning was performed beginning at 20 seconds and 49-71 seconds, respectively. Regions of interest were measured in the head, body, and tail of the pancreas in 92 patients. The difference in enhancement between early and standard delayed scanning was calculated. RESULTS: Mean pancreatic enhancement was 82 HU +/- 3 (standard error) with an early delay, whereas enhancement on standard delay scans was 62 HU +/- 2 (P < .001). An improvement in enhancement greater than 10 HU was attained in 66 of 92 cases (72%). CONCLUSION: Pancreatic enhancement at helical CT with an early delay after contrast material administration is often significantly greater than the enhancement seen with a standard delay when a monophasic, rapidly infused bolus of contrast material is used.     

Ormosil coatings of high hardness

Organically modified silicates (ormosils) of two systems were coated on polyethyleneterephthalate (PET) substrates. One was the tetraethoxysilane (TEOS)-vinyltriethoxysilane (VTES) system and the other was the TEOS-tetraisopropyltitanate (TIPT)-VTES system. The reactions among the alkoxides were examined by liquid state ²⁹Si nuclear magnetic resonance (NMR) spectroscopy. The chemical bonds between TEOS and VTES, between TEOS and TIPT, and between VTES and TIPT are shown in the spectra and the reaction schemes are proposed. Vickers hardnesses of the ormosil coatings were obtained by using the models developed by Jönsson and Hogmark, and the theoretical models developed by the authors were valuable to predict the hardness values. The ormosil coatings obtained in the present study were much harder than the hardest transparent plastics, and thus are very useful for hard or protective coatings on organic polymers.     

Durability of polymeric glazing materials for solar applications

The economic viability of solar collector systems for domestic hot water (DHW) generation is strongly linked to the cost of such systems. Installation and hardware costs must be reduced by 50% to allow significant market penetration. An attractive approach to cost reduction is to replace glass and metal parts with less expensive, lighter weight polymeric components. Weight reduction decreases the cost of shipping, handling, and installation. The use of polymeric materials also allows the benefits and cost savings associated with well established manufacturing processes, along with savings associated with improved fastening, reduced part count, and overall assembly refinements. A key challenge is to maintain adequate system performance and assure requisite durability for extended lifetimes. Results of preliminary and ongoing screening tests for a large number of candidate polymeric glazing materials are presented. Based on these results, two specific glazings with moderate and poor weathering stability are selected to demonstrate how a service lifetime methodology can be applied to accurately predict the optical performance of these materials during in-service use. A summary is given for data obtained by outdoor exposure and indoor testing of polyvinyl chloride (PVC) and high temperature modified polycarbonate copolymer (coPC) materials, and an initial risk analysis is given for the two materials. Screening tests and analyses for service lifetime prediction are discussed. A methodology that provides a way to derive correlations between degradation experienced by materials exposed to controlled accelerated laboratory exposure conditions and materials exposed to in-service conditions is given, and a validation is presented for the methodology based upon durability test results for PVC and coPC.     

Detection rates for abnormal cervical smears: what are we screening for?

We analysed rates of detection for smear abnormalities in 255,000 women served by the Bristol screening programme. The programme began in 1966 with the aim of eradicating the 30-40 deaths each year in Bristol from cervical cancer. Organisation has been good and population uptake has been high for the past 15 years. Records were computerised in 1977. During the 1988 to 1993 screening round, 225,974 women were tested. New smear abnormalities were found in 15,551, of whom nearly 6000 were referred for colposcopy. These numbers are excessively high in comparison with the incidence of the malignancy we are trying to prevent. The effect of screening on death rates in Bristol is too small to detect. Our conclusion is that despite good organisation of the service, much of our effort in Bristol is devoted to limiting the harm done to healthy women and to protecting our staff from litigation as cases of serious disease continue to occur. The real lesson from 30 years' cervical screening is that no matter how obvious the predicted benefit may seem for any screening test, introduction should never take place without adequate prior evaluation of both positive and negative effects in controlled trials.     

Workbench: an AI approach to materials modelling

The Workbench is an adaptable and efficient tool for performing thermodynamic and kinetic calculations. This computer program uses various artificial intelligence techniques to provide a more versatile modelling system than has been possible with conventional programs. The structure and operation of the program is described and a simplified example is worked through illustrating the Workbench environment. Practical applications are given of the use of the Workbench to predict magnetic microstructures in Nd-Fe-B permanent magnets and to study the long term structural stability of Co-Pd multilayers.