When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Electronic Properties of the Layered Perovskite Ruthenates: Correlated Electron Physics Approaching the Low-Disorder Limit

We present a brief review of the physics that is now accessible in the layered perovskite ruthenates that have been the focus of our research for the past few years. The main theme of the work has been the discovery of novel ground states that have an extreme sensitivity to disorder. We discuss these developments and the lessons that they provide for the wider field of correlated electrons in solids.     

Ormosil coatings of high hardness

Organically modified silicates (ormosils) of two systems were coated on polyethyleneterephthalate (PET) substrates. One was the tetraethoxysilane (TEOS)-vinyltriethoxysilane (VTES) system and the other was the TEOS-tetraisopropyltitanate (TIPT)-VTES system. The reactions among the alkoxides were examined by liquid state ²⁹Si nuclear magnetic resonance (NMR) spectroscopy. The chemical bonds between TEOS and VTES, between TEOS and TIPT, and between VTES and TIPT are shown in the spectra and the reaction schemes are proposed. Vickers hardnesses of the ormosil coatings were obtained by using the models developed by Jönsson and Hogmark, and the theoretical models developed by the authors were valuable to predict the hardness values. The ormosil coatings obtained in the present study were much harder than the hardest transparent plastics, and thus are very useful for hard or protective coatings on organic polymers.     

Detection rates for abnormal cervical smears: what are we screening for?

We analysed rates of detection for smear abnormalities in 255,000 women served by the Bristol screening programme. The programme began in 1966 with the aim of eradicating the 30-40 deaths each year in Bristol from cervical cancer. Organisation has been good and population uptake has been high for the past 15 years. Records were computerised in 1977. During the 1988 to 1993 screening round, 225,974 women were tested. New smear abnormalities were found in 15,551, of whom nearly 6000 were referred for colposcopy. These numbers are excessively high in comparison with the incidence of the malignancy we are trying to prevent. The effect of screening on death rates in Bristol is too small to detect. Our conclusion is that despite good organisation of the service, much of our effort in Bristol is devoted to limiting the harm done to healthy women and to protecting our staff from litigation as cases of serious disease continue to occur. The real lesson from 30 years' cervical screening is that no matter how obvious the predicted benefit may seem for any screening test, introduction should never take place without adequate prior evaluation of both positive and negative effects in controlled trials.     

Workbench: an AI approach to materials modelling

The Workbench is an adaptable and efficient tool for performing thermodynamic and kinetic calculations. This computer program uses various artificial intelligence techniques to provide a more versatile modelling system than has been possible with conventional programs. The structure and operation of the program is described and a simplified example is worked through illustrating the Workbench environment. Practical applications are given of the use of the Workbench to predict magnetic microstructures in Nd-Fe-B permanent magnets and to study the long term structural stability of Co-Pd multilayers.     

A Multi-Criteria Decision Analysis Technique for Stochastic Task Criticality in Project Management

Inherent in project management is the risk that a project fails to meet planned completion deadlines due to delays experienced in individual tasks. As such, certain critical tasks may be candidates for risk management (e.g., the allocation of additional resources such as labor, materials, and equipment) to prevent delays. A common means to identify such critical tasks is with the critical path method (CPM), which identifies a path of tasks in a project network that, when delayed, result in project delays. This work offers a complementary, stochastic approach to CPM that ranks tasks according to their effect on the project completion time distribution, when the distributions of task completion time are delayed. The new hybrid approach is based on the use of a Monte Carlo simulation and a multi-criteria decision analysis technique. Monte Carlo simulation allows for approximating the cumulative distribution function of the total duration of the project, while the multi-criteria decision analysis technique is used to compare and rank the tasks across percentiles of the resulting project completion time distributions. Doing so allows for different percentile weighting schemes to represent decision maker risk preferences. The suggested approach is applied to two project network examples. The examples illustrate that the proposed approach highlights some tasks as risky, which may not always lie on the critical path as identified by CPM. This is valuable for practicing managers as it allows them to properly consider their risk preferences when determining task criticality based on the distribution of project completion time (e.g., emphasizing median vs. upper tail completion time).     

Preparation and non-linear optical properties of CdS quantum dots in Na2O-B2O3-SiO2 glasses by the sol-gel technique

Non-linear optical properties of CdS quantum dots embedded in the sol-gel derived Na₂O-B₂O₃-SiO₂x³ glass matrix have been studied using nanosecond degenerate four-wave mixing (DFWM), pump-probe experiments, and time-resolved subpicosecond DFWM measurements. The concentration of CdS microcrystallites was varied from 1.4 to 10.2 wt% while the particle size was controlled to be in the range of 3–6 nm in diameter so that the confinement effects can be realized. The third-order susceptibility, ³, was determined to be of the order of 10^–7-10^–6 e.s.u. near the resonant wavelength between 450 and 470 nm by the nanosecond DFWM and pump-probe experiments, and of the order of 10^–11-10^–10 e.s.u. at the off-resonant wavelength, 580 nm, by a time-resolved DFWM measurement with 400 fs laser pulse. The decay time changed from 0.5 to 50 picoseconds as a function of the size, size distribution and number density of CdS microcrystallites in the glasses. Photodarkening reduced the optical non-linearity of the melt-quenched samples by a factor of 20, while it had no appreciable effect on that of the sol-gel derived samples.[/p]     

Predicting Mechanical Strength of In-Use Firefighter Protective Clothing Using Near-Infrared Spectroscopy

The exact lifespan of in-use firefighter protective clothing is difficult to predict due to the large variations in use between individual garments. Furthermore, testing methods used to evaluate new protective clothing are destructive in nature and could not be applied to in-use garments. Various non-destructive techniques have been proposed for the evaluation of in-use clothing, each possessing its own advantages and disadvantages. The ability of near-infrared spectroscopy to predict the tensile strength of thermally aged fabrics used in protective clothing for wildland firefighters and other workers is investigated here. Fabrics were exposed to heat fluxes from 10 kW/m² to 40 kW/m² for various durations using the cone calorimeter, after which the tensile strength of the fabrics was measured. Temperatures measured during the exposures and results of thermal gravimetric analysis tests were used to interpret changes in tensile strength. Multivariate linear regression was used to develop correlations between the tensile strength and the reflectance values measured between 1500 nm and 2500 nm for new and thermally aged fabrics. It was found that models based on reflectance measurements made at as few as three wavelengths could be used to estimate the tensile strength of the thermally aged specimens.