When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

A concept for the generation of out-of-plane distortion from tailored FRP laminates

The phenomenon of thermally induced distortions in unsymmetrical laminates is well understood, and it may be shown that a square, unsymmetrical 0.90 laminate will tend to form two stable geometries with a snap-through phenomenon between them. This paper discusses laminates in which at each point the lay-up is symmetrical across the laminate mid-plane, but which still exhibit multiple stable geometries. The number of stable geometries can be controlled by the details of the lay-up from the minimum of two to, in principle, an unlimited number. In addition it will be shown how a similar process can be used to generate multiple stable stress states and geometries in unidirectional laminates. This paper represents a very preliminary experimental investigation of the design space available for such composite laminates. Possible applications of composites with multiple stable geometries are noted.     

Specification methods for material-handling control algorithms in flexible manufacturing systems

Good methods are needed to specify, test, and debug material-handling control logic. This article surveys a number of representative methods for defining and describing control algorithms for programmable material-handling equipment used in flexible manufacturing systems. The methods are evaluated with regard to their suitability for communication between people and as bases for interfaces to automatic program generators. It is concluded that no single method is entirely satisfactory. Three methods (position diagrams, function block diagrams, and operation networks) have potential to be combined into an effective hybrid approach that minimizes the need for the user to switch between various conceptual models.     

Arc root mobility on piezoelectrically actuated contacts in miniature circuit breakers

A novel contact opening mechanism has been developed using a piezoelectric actuator to open the contacts in a low contact opening velocity circuit breaker. The arc control on the contacts is critical for successful current interruption (10/sup 3/-10/sup 4/ A) in low voltage (<250V) devices. Previous work has shown how arc root commutation from the contact region into the arc chamber is affected by arc chamber materials, contact materials and the gap behind the moving contact for contact velocities between 1ms/sup -1/ and 10ms/sup -1/. This work is extended using a commercially available piezoelectric actuator to open the contacts. Contact opening speeds are assessed and the arc root mobility is characterized under this operating regime. A flexible test apparatus and solid-state high-speed arc imaging system are used to gather data on the arc root during the opening of the contacts. New experimental results are presented on the anode and cathode root velocity and arc root motion in an arc chamber with piezoelectrically actuated contact opening. These results can be used to improve the design of high current low voltage circuit breakers suitable for piezoelectric actuation.     

High-dose busulfan, melphalan, thiotepa and peripheral blood stem cell infusion for the treatment of metastatic breast cancer

The purpose of this study was to determine the outcome of patients with metastatic breast cancer treated with high-dose busulfan (Bu), melphalan (Mel) and thiotepa (TT) followed by peripheral blood stem cell (PBSC) infusion. Fifty-one patients with chemotherapy refractory (n = 32) or responsive (n = 19) metastatic breast cancer received Bu (12 mg/kg), Mel (100 mg/m2) and TT (500 mg/m2) followed by PBSC collected after chemotherapy and growth factor (n = 43) or growth factor alone (n = 8). The 100 day treatment-related mortality was 8% including one death from cytomegalovirus pneumonia, one from aspiration pneumonia and two from regimen-related toxicity (RRT). Seven of 28 refractory (25%) and 5/7 (71%) responsive patients with evaluable disease achieved a complete response of all measurable disease or all soft tissue disease with at least improvement in bone lesions (PR*). Fifteen of 51 patients (29%) are alive and progression-free a median of 423 days (range 353-934) after treatment, 5/32 (16%) with refractory disease and 10/19 (53%) with responsive disease. The probabilities of progression-free survival (PFS) at 1.5 years for the patients with refractory (n = 32) and responsive (n = 19) disease were 0.24 and 0.53, respectively. These preliminary data suggest that high-dose Bu/Mel/TT has significant activity in patients with advanced breast cancer and may be superior to some previously published regimens.     

A process oriented semantics of the PRAM-language FORK

The parallel language FORK [1], based on a scalable shared memory model, is a PASCAL-like language with some additional parallel constructs. A PRAM (Parallel Random Access Machine) algorithm can be expressed on a high level of abstraction as a FORK program which is translated into efficient PRAM code guaranteeing theoretically predicted runtimes.

In this paper, we concentrate on those features of the language FORK related to parallelism, such as the group concept, a shared memory access and synchronous or asynchronous execution. We present a trace-based denotational interleaving semantics where processes describe synchronous computations. Processes are created or deleted dynamically and run asynchronously. Interleaving rules reflect the underlying CRCW (concurrent-read-concurrent-write) PRAM model.     

Uncertainty in Quantitative Electron Probe Microanalysis

Quantitative electron probe analysis is based on models based on the physics or x-ray generation, empirically adjusted to the analyses of specimens of known composition. Their accuracy can be estimated by applying them to a set of specimens of presumably well-known composition.     

Perceived risk for influenza in veterans with spinal cord injuries and disorders.

Objective: Guided by the extended parallel process model (EPPM), the objective was to assess control processes dominant in influenza behavior decisions. Design: Cross-sectional survey. Results: Response rate was 31% (n=968). Regarding influenza risk, 59% were in danger control. Those in fear control were more likely to report influenza infection (p=.000). In the nonvaccinated, those in fear control were more likely to indicate not knowing where to get the vaccine (p=.016) and that it was unavailable (p=.027), and those in danger control believed they did not need it (p=.023). Zero critical values were more likely to indicate that no health provider recommended the shot (p=.002). Conclusions: Most perceived efficacy to be stronger than threat related to influenza; according to the EPPM, they are aware of their risk but recognize their ability to avert it. For those in danger control, messages should focus on increasing perceptions of severity and susceptibility to positively affect behavior change. For those in fear control, messages should focus on efficacy only. With a critical value of zero, no threat is induced, and a high-threat/high-efficacy approach should be taken. (PsycINFO Database Record (c) 2010 APA, all rights reserved)