When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Multicellular Tumor Spheroids (MCTS) as a 3D In Vitro Evaluation Tool of Nanoparticles

Multicellular tumor spheroid models (MCTS) are often coined as 3D in vitro models that can mimic the microenvironment of tissues. MCTS have gained increasing interest in the nano‐biotechnology field as they can provide easily accessible information on the performance of nanoparticles without using animal models. Considering that many countries have put restrictions on animals testing, which will only tighten in the future as seen by the recent developments in the Netherlands, 3D models will become an even more valuable tool. Here, an overview on MCTS is provided, focusing on their use in cancer research as most nanoparticles are tested in MCTS for treatment of primary tumors. Thereafter, various types of nanoparticles—from self‐assembled block copolymers to inorganic nanoparticles, are discussed. A range of physicochemical parameters including the size, shape, surface chemistry, ligands attachment, stability, and stiffness are found to influence nanoparticles in MCTS. Some of these studies are complemented by animal studies confirming that lessons from MCTS can in part predict the behaviour in vivo. In summary, MCTS are suitable models to gain additional information on nanoparticles. While not being able to replace in vivo studies, they can bridge the gap between traditional 2D in vitro studies and in vivo models.     

Specification methods for material-handling control algorithms in flexible manufacturing systems

Good methods are needed to specify, test, and debug material-handling control logic. This article surveys a number of representative methods for defining and describing control algorithms for programmable material-handling equipment used in flexible manufacturing systems. The methods are evaluated with regard to their suitability for communication between people and as bases for interfaces to automatic program generators. It is concluded that no single method is entirely satisfactory. Three methods (position diagrams, function block diagrams, and operation networks) have potential to be combined into an effective hybrid approach that minimizes the need for the user to switch between various conceptual models.     

A process oriented semantics of the PRAM-language FORK

The parallel language FORK [1], based on a scalable shared memory model, is a PASCAL-like language with some additional parallel constructs. A PRAM (Parallel Random Access Machine) algorithm can be expressed on a high level of abstraction as a FORK program which is translated into efficient PRAM code guaranteeing theoretically predicted runtimes.

In this paper, we concentrate on those features of the language FORK related to parallelism, such as the group concept, a shared memory access and synchronous or asynchronous execution. We present a trace-based denotational interleaving semantics where processes describe synchronous computations. Processes are created or deleted dynamically and run asynchronously. Interleaving rules reflect the underlying CRCW (concurrent-read-concurrent-write) PRAM model.     

Uncertainty in Quantitative Electron Probe Microanalysis

Quantitative electron probe analysis is based on models based on the physics or x-ray generation, empirically adjusted to the analyses of specimens of known composition. Their accuracy can be estimated by applying them to a set of specimens of presumably well-known composition.     

Biocide polymers

Summary Potential polymeric herbicides have been synthesized by reacting poly(acrylic acid)s with 3-aniino-1,2,4-triazole and 1-acetyl-5-amino-1,2,4-triazole, respectively. The produced copolymers contained the active moieties as pendant groups in the range of 3 to 27 per cent of weight and with a content of carboxyl groups up to 95 mol-%. The hydrolytic release of 3-amino-1,2,4-triazole was investigated.Part 12: M. Hartmann, H,-J. Bauer, A. Pritzsche and K. Wermann, Makromol. Chem., in press     

Computer-aided design in mechanisms

The man-machine dialog is very important for the design process. In this paper a program system, based on the minicomputer KRS 4201 (ROBOTRON), is demonstrated by using an interactive graphical screen. This program system provides the opportunity to analyze and synthesize mechanisms up to 16 links. The application is demonstrated by designing a front-loader-system for a truck. Real-time animation of seven-bar linkages and coupler curves, including velocity and acceleration, are shown.     

Injection mold flashing of liquid crystalline polymers

Thermotropic polyesters, such as Vectra (Hoechst Celanese), have excellent moldability for intricate parts that require high precision of form, such as electronic connectors. Two apparently contradictory aspects of molding behavior contribute to the moldability. On the one hand, the low viscosity of the liquid crystalline polymer (LCP) at high shear rates favors ease of filling molds that contain long, thin paths. On the other, parts molded from LCP have little or no flash to interfere with the functioning of the parts. There has apparently been little work on the rheological aspects of flash formation. An approximate analysis is made by considering that the flash is the result of melt being extruded from the mold cavity into a slit at the mold parting line. The driving force for the extrusion is the injection pressure. The flow is assumed to be isothermal until solidification occurs, at a time that depends on the thickness of the slit, on the thermal diffusivity of the melt, the melt and mold temperatures, and on the solidification temperature of the material. The viscosity is assumed to have power-law dependence on shear rate. It is found that when the aspect ratio (length to thickness) of the flash is small, its length is strongly dependent on the magnitude of the pressure drop at the contraction from the cavity to the slit. At the minimum pressure required to fill a mold, the flash length is predicted to be independent of the rheological and thermal properties of the melt, except for the power-law exponent. Differences in end correction can, however, account for different tendencies to flash at equal moldability. Comparison of the model with Richardson's analysis of freezing in a cavity suggests a correlation of the thermal properties of the melt with his parameter c, which is related to mold filling ability. Tests of the model and possible refinements are suggested.     

Influence of particles on recrystallization textures of ferritic stainless steels

Five microalloyed ferritic steels with a Cr content of 10 to 17 % were cold rolled and recrystallized. Steels with a high volume fraction of Nb and Ti precipitations revealed a deviation from the typical {111} fibre texture. The new main component {557}<583> was explained by particle induced growth selection.