When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Specification methods for material-handling control algorithms in flexible manufacturing systems

Good methods are needed to specify, test, and debug material-handling control logic. This article surveys a number of representative methods for defining and describing control algorithms for programmable material-handling equipment used in flexible manufacturing systems. The methods are evaluated with regard to their suitability for communication between people and as bases for interfaces to automatic program generators. It is concluded that no single method is entirely satisfactory. Three methods (position diagrams, function block diagrams, and operation networks) have potential to be combined into an effective hybrid approach that minimizes the need for the user to switch between various conceptual models.     

Effects of sex steroids on sensory and motor spinal mechanisms

The male copulatory pattern uses muscles in the penis for erection and penile insertion, the lower trunk for pelvic thrusting, and the sex accessory organs for seminal emission. Organization of the nuclei controlling penile muscles is achieved through cell growth, dendritic arborization, and synaptogenesis, actions dependent on androgen but not estrogen. Testosterone (T) and dihydrotestosterone (DHT) but not estradiol (E2), stimulate pelvic thrusting vigor by synchronizing discharge of motoneurons innervating pelvic muscles. Pelvic thrusting rhythmicity, regulated by spinal interneurons, is produced in female rabbits by E2 or T but not by DHT. Reflex contraction of the seminal vesicles, due to penile insertion, is facilitated by androgen presumably by its effect on preganglionic neurons of the hypogastric nerve, located in the dorsal commissural nucleus.     

Structural properties and electrical behaviour of thin silicon oxynitride layers

Thin SiO₂ and SiO_xN_y layers were grown on silicon using Rapid Thermal Processing (RTP) in either O₂ or N₂O ambient. Subsequent annealing or nitridation was performed in order to improve the electrical stability. The composition of the films, in particular the incorporation of nitrogen and hydrogen, has been studied. We obtained the distribution of states at the Si/insulator interface through the evaluation of CV measurements and investigated the charge trapping in the layers analysing the voltage–time behaviour during Fowler–Nordheim constant current injection. Furthermore, assuming a trap assisted tunneling mechanism, the influence of near interface trap states on the current voltage characteristic was used to derive an effective insulator state distribution.     

A process oriented semantics of the PRAM-language FORK

The parallel language FORK [1], based on a scalable shared memory model, is a PASCAL-like language with some additional parallel constructs. A PRAM (Parallel Random Access Machine) algorithm can be expressed on a high level of abstraction as a FORK program which is translated into efficient PRAM code guaranteeing theoretically predicted runtimes.

In this paper, we concentrate on those features of the language FORK related to parallelism, such as the group concept, a shared memory access and synchronous or asynchronous execution. We present a trace-based denotational interleaving semantics where processes describe synchronous computations. Processes are created or deleted dynamically and run asynchronously. Interleaving rules reflect the underlying CRCW (concurrent-read-concurrent-write) PRAM model.     

Uncertainty in Quantitative Electron Probe Microanalysis

Quantitative electron probe analysis is based on models based on the physics or x-ray generation, empirically adjusted to the analyses of specimens of known composition. Their accuracy can be estimated by applying them to a set of specimens of presumably well-known composition.     

Putting it all together – Formal verification of the VAMP

In the verified architecture microprocessor (VAMP) project we have designed, functionally verified, and synthesized a processor with full DLX instruction set, delayed branch, Tomasulo scheduler, maskable nested precise interrupts, pipelined fully IEEE compatible dual precision floating point unit with variable latency, and separate instruction and data caches. The verification has been carried out in the theorem proving system PVS. The processor has been implemented on a Xilinx FPGA. A shorter version of this article with the title “Instantiating uninterpreted functional units and memory system: functional verification of the VAMP” appeared in [8]. The work reported here was done while all the authors were with Saarland University.     

Biocide polymers

Summary Potential polymeric herbicides have been synthesized by reacting poly(acrylic acid)s with 3-aniino-1,2,4-triazole and 1-acetyl-5-amino-1,2,4-triazole, respectively. The produced copolymers contained the active moieties as pendant groups in the range of 3 to 27 per cent of weight and with a content of carboxyl groups up to 95 mol-%. The hydrolytic release of 3-amino-1,2,4-triazole was investigated.Part 12: M. Hartmann, H,-J. Bauer, A. Pritzsche and K. Wermann, Makromol. Chem., in press