Interactive visualization for testing Markov Decision Processes: MDPVIS |
| |
| Affiliation: | 1. School of Electrical Engineering and Computer Science, Oregon State University, 1148 Kelley Engineering Center, Corvallis, OR 97331-4501, USA;2. Department of Forest Engineering, Oregon State University, USA;3. Computer Science and Engineering, University of Notre Dame, USA;1. Apple Inc., Cupertino, CA, USA;2. ABB Corporate Research, Raleigh, NC, USA;3. Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA;4. Department of Computer Science, North Carolina State University, Raleigh, NC, USA;1. Human Centred Design Institute, Brunel University London, London, UK;2. HCI Centre, School of Computer Science, University of Birmingham, Birmingham, UK;3. Talis, Birmingham, UK;1. School of EECS, Oregon State University, Corvallis, OR, USA;2. Information Systems, New Jersey Institute of Technology, Newark, NJ, USA;3. Information School, University of Washington, Seattle, WA, USA |
| |
| Abstract: | Markov Decision Processes (MDPs) are a formulation for optimization problems in sequential decision making. Solving MDPs often requires implementing a simulator for optimization algorithms to invoke when updating decision making rules known as policies. The combination of simulator and optimizer are subject to failures of specification, implementation, integration, and optimization that may produce invalid policies. We present these failures as queries for a visual analytic system (MDPVIS). MDPVIS addresses three visualization research gaps. First, the data acquisition gap is addressed through a general simulator-visualization interface. Second, the data analysis gap is addressed through a generalized MDP information visualization. Finally, the cognition gap is addressed by exposing model components to the user. MDPVIS generalizes a visualization for wildfire management. We use that problem to illustrate MDPVIS and show the visualization's generality by connecting it to two reinforcement learning frameworks that implement many different MDPs of interest in the research community. |
| |
| Keywords: | Visualization Markov decision process Testing Parameter space analysis Wildfire Optimization |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
