When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Deep Learning Based Process Analytics Model for Predicting Type 2 Diabetes Mellitus

Process analytics is one of the popular research domains that advanced in the recent years. Process analytics encompasses identification, monitoring, and improvement of the processes through knowledge extraction from historical data. The evolution of Artificial Intelligence (AI)-enabled Electronic Health Records (EHRs) revolutionized the medical practice. Type 2 Diabetes Mellitus (T2DM) is a syndrome characterized by the lack of insulin secretion. If not diagnosed and managed at early stages, it may produce severe outcomes and at times, death too. Chronic Kidney Disease (CKD) and Coronary Heart Disease (CHD) are the most common, long-term and life-threatening diseases caused by T2DM. Therefore, it becomes inevitable to predict the risks of CKD and CHD in T2DM patients. The current research article presents automated Deep Learning (DL)-based Deep Neural Network (DNN) with Adagrad Optimization Algorithm i.e., DNN-AGOA model to predict CKD and CHD risks in T2DM patients. The paper proposes a risk prediction model for T2DM patients who may develop CKD or CHD. This model helps in alarming both T2DM patients and clinicians in advance. At first, the proposed DNN-AGOA model performs data preprocessing to improve the quality of data and make it compatible for further processing. Besides, a Deep Neural Network (DNN) is employed for feature extraction, after which sigmoid function is used for classification. Further, Adagrad optimizer is applied to improve the performance of DNN model. For experimental validation, benchmark medical datasets were used and the results were validated under several dimensions. The proposed model achieved a maximum precision of 93.99%, recall of 94.63%, specificity of 73.34%, accuracy of 92.58%, and F-score of 94.22%. The results attained through experimentation established that the proposed DNN-AGOA model has good prediction capability over other methods.     

减少手术创伤始终是妇科肿瘤手术快速康复的决定性因素

减少手术创伤始终是快速康复的决定性因素,这一点在目前的加速康复外科研究尤其是复杂手术,如妇科肿瘤手术中尚未得到充分重视。尊重学习曲线、全面规划手术方案、总结失利经验、开展前瞻性研究是解决此问题的主要方案。本文着重讨论妇科肿瘤手术创伤对术后加速康复的影响及可能的改进措施。     

Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

A knowledge construction methodology to automate case-based learning using clinical documents

The case-based learning (CBL) approach has gained attention in medical education as an alternative to traditional learning methodology. However, current CBL systems do not facilitate and provide computer-based domain knowledge to medical students for solving real-world clinical cases during CBL practice. To automate CBL, clinical documents are beneficial for constructing domain knowledge. In the literature, most systems and methodologies require a knowledge engineer to construct machine-readable knowledge. Keeping in view these facts, we present a knowledge construction methodology (KCM-CD) to construct domain knowledge ontology (i.e., structured declarative knowledge) from unstructured text in a systematic way using artificial intelligence techniques, with minimum intervention from a knowledge engineer. To utilize the strength of humans and computers, and to realize the KCM-CD methodology, an interactive case-based learning system(iCBLS) was developed. Finally, the developed ontological model was evaluated to evaluate the quality of domain knowledge in terms of coherence measure. The results showed that the overall domain model has positive coherence values, indicating that all words in each branch of the domain ontology are correlated with each other and the quality of the developed model is acceptable.     

Quantitative investigation of electromechanical coupling of potassium sodium niobate-based thin films

High-performance environment-friendly piezoelectric potassium sodium niobate (KNN)-based thin films have been emerged as promising lead-free candidates, while their substrate-dependent piezoelectricity faces the lack of high-quality information due to restraints in measurements. Although piezoresponse force microscopy (PFM) is a potential measuring tool, still its regular mode is not considered as a reliable characterization method for quantification. After combining machine-learning enabled analysis using PFM datasets, it is possible to measure piezoelectric properties quantitatively. Here we utilized advanced PFM technology empowered by machine learning to measure and compare the piezoelectricity of KNN based thin films on different substrates. The results provide a better understanding of the relationship between structures and piezoelectric properties of the thin films.     

Usability of a cloud-based collaborative learning framework to improve learners’ experience

Computer-Supported Collaborative Learning (CSCL) is concerned with how Information and Communication Technology (ICT) might facilitate learning in groups which can be co-located or distributed over a network of computers such as Internet. CSCL supports effective learning by means of communication of ideas and information among learners, collaborative access of essential documents, and feedback from instructors and peers on learning activities. As the cloud technologies are increasingly becoming popular and collaborative learning is evolving, new directions for development of collaborative learning tools deployed on cloud are proposed. Development of such learning tools requires access to substantial data stored in the cloud. Ensuring efficient access to such data is hindered by the high latencies of wide-area networks underlying the cloud infrastructures. To improve learners’ experience by accelerating data access, important files can be replicated so a group of learners can access data from nearby locations. Since a cloud environment is highly dynamic, resource availability, network latency, and learner requests may change. In this paper, we present the advantages of collaborative learning and focus on the importance of data replication in the design of such a dynamic cloud-based system that a collaborative learning portal uses. To this end, we introduce a highly distributed replication technique that determines optimal data locations to improve access performance by minimizing replication overhead (access and update). The problem is formulated using dynamic programming. Experimental results demonstrate the usefulness of the proposed collaborative learning system used by institutions in geographically distributed locations.     

基于虚拟化技术的在线开放课程建设研究--以“常用工具软件”课程为例

随着互联网的迅猛发展,建设在线开放课程成为高职院校信息化教学改革的一个方向。首先通过分析国内外的在线课程现状,发现阻碍在线课程发展的共性问题;然后对"常用工具软件"课程现状进行深入调查、分析和研究,提出将虚拟化技术应用到课程建设中以解决网络教学遇到的问题;最后将研究结论应用到教学实践中,用于提升学生专业能力,提高教学效果。     

Lecture capture with real‐time rearrangement of visual elements: impact on student performance

The primary goal of this study is to create and test a lecture‐capture system that can rearrange visual elements while recording is still taking place, in such a way that student performance can be positively influenced. The system we have devised is capable of integrating and rearranging multimedia sources, including learning content, the instructor and students' images, into lecture videos that are embedded in a website for students to review after school. The present study employed a two‐group experimental design, with 153 participants (145 females and 8 males) making up an experimental group in which lecture courses were recorded using the new lecture‐capture system, and 149 participants (140 females and 9 males) forming a control group whose lectures were recorded by traditional means. All participants were in the freshman college and studying Introduction to Computer and Information Science in one of six classes, and were randomly assigned to one of the two groups. The participants' midterm examination and final examination scores were collected as indicators of their academic performance, with their mathematics entrance scores used as a pre‐test. The findings obtained from analysis of covariance (ANCOVA) suggest that appropriate rearrangement of visual elements in lecture videos can significantly impact students' learning performance.     

ProVoc: An app to train vocabulary depth in order to foster children's reading comprehension

Many studies have demonstrated the crucial role of vocabulary in predicting reading performance in general. More recent work has indicated that one particular facet of vocabulary (its depth) is more closely related to language comprehension, especially inferential comprehension. On this basis, we developed a training application to specifically improve vocabulary depth. The objective of this study was to test the effectiveness of a mobile application designed to improve vocabulary depth. The effectiveness of this training was examined on 3rd and 4th grade children's vocabulary (breadth and depth), decoding and comprehension performances. A randomized waiting-list control paradigm was used in which an experimental group first received the intervention during the first 4 weeks (between pretest and post-test1), thereafter, a waiting control group received the training for the next 4 weeks (between postest1 and posttest2). Results showed that the developed application led to significant improvements in terms of vocabulary depth performance, as well as a significant transfer effect to reading comprehension. However, we did not observe such a beneficial effect on either vocabulary breadth or written word identification. These results are discussed in terms of the links between vocabulary depth and comprehension, and the opportunities the app presents for remedying language comprehension deficits in children.