When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Serum antibody opsonic activity against Actinobacillus actinomycetemcomitans in human periodontal diseases

Actinobacillus actinomycetemcomitans is frequently associated with severe periodontitis. Many periodontitis patients have elevated levels of serum IgG antibodies to A. actinomycetemcomitans, but the role of these antibodies is unknown. This study evaluated the functional capacity of anti-A. actinomycetemcomitans IgG antibody to enhance phagocytosis of A. actinomycetemcomitans by polymorphonuclear leukocytes. Chemoluminescence assays were done using sera from 64 subjects, 61 of whom had severe periodontitis; results were compared with the subject's anti-A. actinomycetemcomitans IgG titer and avidity. There was a strong correlation between chemoluminescence and antibody log titer (P < .00001) and a weak correlation between chemoluminescence and antibody avidity (P < .05). The results support the hypothesis that anti-A. actinomycetemcomitans IgG antibodies are important in promoting phagocytosis and killing of A. actinomycetemcomitans. Subjects who develop high levels of highly avid antibodies against A. actinomycetemcomitans may have greater resistance to continued or repeated infection by this pathogen.     

Specification methods for material-handling control algorithms in flexible manufacturing systems

Good methods are needed to specify, test, and debug material-handling control logic. This article surveys a number of representative methods for defining and describing control algorithms for programmable material-handling equipment used in flexible manufacturing systems. The methods are evaluated with regard to their suitability for communication between people and as bases for interfaces to automatic program generators. It is concluded that no single method is entirely satisfactory. Three methods (position diagrams, function block diagrams, and operation networks) have potential to be combined into an effective hybrid approach that minimizes the need for the user to switch between various conceptual models.     

A process oriented semantics of the PRAM-language FORK

The parallel language FORK [1], based on a scalable shared memory model, is a PASCAL-like language with some additional parallel constructs. A PRAM (Parallel Random Access Machine) algorithm can be expressed on a high level of abstraction as a FORK program which is translated into efficient PRAM code guaranteeing theoretically predicted runtimes.

In this paper, we concentrate on those features of the language FORK related to parallelism, such as the group concept, a shared memory access and synchronous or asynchronous execution. We present a trace-based denotational interleaving semantics where processes describe synchronous computations. Processes are created or deleted dynamically and run asynchronously. Interleaving rules reflect the underlying CRCW (concurrent-read-concurrent-write) PRAM model.     

Uncertainty in Quantitative Electron Probe Microanalysis

Quantitative electron probe analysis is based on models based on the physics or x-ray generation, empirically adjusted to the analyses of specimens of known composition. Their accuracy can be estimated by applying them to a set of specimens of presumably well-known composition.     

Comparison of scene segmentations: SMAP, ECHO, and maximumlikelihood

Sequential maximum a posteriori (SMAP) and the extraction and classification of homogeneous objects (ECHO), two spectral/spatial scene segmentation algorithms, were compared with traditional maximum likelihood (ML) estimation in a supervised classification of multispectral data. SMAP generalized better than both ECHO and ML. Significant differences were found in all mean class classification accuracies: SMAP>ECHO>ML     

An analysis of interleukin-8, interleukin-6 and C-reactive protein serum concentrations to predict fever, gram-negative bacteremia and complicated infection in neutropenic cancer patients

A prospective study was performed to assess the potential value of interleukin (IL)-8, IL-6, and C-reactive protein (CRP) serum levels to predict fever, gram-negative bacteremia and complicated infection in neutropenic patients with cancer. Serum samples were obtained three times a week during 208 neutropenic episodes following cytotoxic chemotherapy. Fever of any cause developed during 104 out of 191 evaluable episodes. Serum levels of neither cytokine nor CRP were predictive of fever within more than 24 h before its onset. Unlike CRP, both IL-6 and IL-8 serum levels were significantly different between microbiologically documented infections and unexplained fevers. The highest values of IL-6 and IL-8 were observed in episodes of gram-negative bacteremia. Using receiver-operating-characteristic curves, the analysis of cytokine levels measured around the onset of fever indicated that IL-8 is potentially useful for predicting gram-negative bacteremia, with a high negative predictive value of > 90% and a moderate positive predictive value of 50% (sensitivity, 70%; specificity, 91%). In patients with persistent fever, predictions of further clinical complications, defined as prolonged fever of more than 7 days' duration, pneumonia, shock and/or death due to infection, were best predicted by IL-6. With an IL-6 cutoff value of 250 pg/ml in samples obtained 8 to 32 h after onset of fever, the positive predictive value was 92%, the negative predictive value 91% (sensitivity, 85%; specificity, 95%). The positive predictive value of IL-6 in samples obtained another 24 h later from patients still febrile remained > 90%, but the negative predictive value dropped to 47%. In any of the analyses, the predictive values of CRP levels were poor and inferior to either cytokine. These findings may have clinical value in identifying subgroups of patients requiring different therapeutic approaches.     

Buchbesprechungen

Ohne Zusammenfassung

---

Book reviews