Qualitative reachability in stochastic BPA games

We consider a class of infinite-state stochastic games generated by stateless pushdown automata (or, equivalently, 1-exit recursive state machines), where the winning objective is specified by a regular set of target configurations and a qualitative probability constraint ‘>0’ or ‘=1’. The goal of one player is to maximize the probability of reaching the target set so that the constraint is satisfied, while the other player aims at the opposite. We show that the winner in such games can be determined in P for the ‘>0’ constraint, and in NP∩co-NP for the ‘=1’ constraint. Further, we prove that the winning regions for both players are regular, and we design algorithms which compute the associated finite-state automata. Finally, we show that winning strategies can be synthesized effectively.     

Risk-Sensitive and Robust Escape Control for Degenerate Diffusion Processes

This paper considers the problem of controlling a possibly degenerate small noise diffusion so as to prevent it from leaving a prescribed set. The criterion of interest is a risk-sensitive version of the mean escape time criterion. Using a general representation formula, this criterion is expressed as the upper value of a stochastic differential game. It is shown that in the small noise limit this upper value converges to the value of an associated deterministic differential game. Our approach differs from standard PDE approaches in a number of ways. For example, the upper game representation allows one to relate directly the prelimit and the limit controls and, in fact, strategies that are nearly maximizing for the robust problem can be used to define nearly minimizing controls for the risk-sensitive control problem for sufficiently small ε>0. The result provides a canonical example of the use of variational representations in connecting risk-sensitive and robust control. Date received: November 21, 1998. Date revised: June 20, 1999.     

<Emphasis Type="Italic">H</Emphasis><Subscript>∞</Subscript> suboptimal tracking controller design for a class of nonlinear systems

In this paper, a new technique is proposed to solve the H _∞ tracking problem for a broad class of nonlinear systems. Towards this end, based on a discounted cost function, a nonlinear two-player zero-sum differential (NTPZSD) game is defined. Then, the problem is converted to another NTPZSD game without any discount factor in its corresponding cost function. A state-dependent Riccati equation (SDRE) technique is applied to the latter NTPZSD game in order to find its approximate solution which leads to obtain a feedback-feedforward control law for the original game. It is proved that the tracking error between the system state and its desired trajectory converges asymptotically to zero under mild conditions on the discount factor. The proposed H _∞ tracking controller is applied to two nonlinear systems (the Vander Pol’s oscillator and the insulin-glucose regulatory system of type I diabetic patients). Simulation results demonstrate that the proposed H _∞ tracking controller is so effective to solve the problem of tracking time-varying desired trajectories in nonlinear dynamical systems.     

Effects of constructing versus playing an educational game on student motivation and deep learning strategy use

In this study the effects of two different interactive learning tasks, in which simple games were included were described with respect to student motivation and deep strategy use. The research involved 235 students from four elementary schools in The Netherlands. One group of students (N = 128) constructed their own memory ‘drag and drop’ game, whereas the other group (N = 107) played an existing ‘drag and drop’ memory game. Analyses of covariance demonstrated a significant difference between the two conditions both on intrinsic motivation and deep strategy use. The large effect sizes for both motivation and deep strategy use were in favour of the construction condition. The results suggest that constructing a game might be a better way to enhance student motivation and deep learning than playing an existing game. Despite the promising results, the low level of complexity of the games used is a study limitation.     

Finite Horizon Minimax Optimal Control of Stochastic Partially Observed Time Varying Uncertain Systems

We consider a linear-quadratic problem of minimax optimal control for stochastic uncertain control systems with output measurement. The uncertainty in the system satisfies a stochastic integral quadratic constraint. To convert the constrained optimization problem into an unconstrained one, a special S-procedure is applied. The resulting unconstrained game-type optimization problem is then converted into a risk-sensitive stochastic control problem with an exponential-of-integral cost functional. This is achieved via a certain duality relation between stochastic dynamic games and risk-sensitive stochastic control. The solution of the risk-sensitive stochastic control problem in terms of a pair of differential matrix Riccati equations is then used to establish a minimax optimal control law for the original uncertain system with uncertainty subject to the stochastic integral quadratic constraint. Date received: May 13, 1997. Date revised: March 18, 1998.     

Development of a scale for fantasy state in digital games

Digital games appear to motivate players intrinsically. Of various game features, fantasy in particularly plays a crucial role in enhancing motivation and is a key factor in immersion in gameplay. As with its inherent value, fantasy also plays a vital role in distinguishing digital games itself from other media. Despite its significance, fantasy has received little attention, and this concept is still ambiguous to define with any certainty. This study thus aims to create a framework to explore a dimension of fantasy and to develop a scale to measure a state of fantasy in digital games. As a result, four factors were extracted, which were ‘identification’, ‘imagination’, ‘analogy’, and ‘satisfaction’, to account for fantasy state in digital gameplay. Based on these factors, a fantasy scale in digital games (FSDGs) included 16 items was developed.     

Robust output feedback model predictive control of constrained linear systems

This paper provides a solution to the problem of robust output feedback model predictive control of constrained, linear, discrete-time systems in the presence of bounded state and output disturbances. The proposed output feedback controller consists of a simple, stable Luenberger state estimator and a recently developed, robustly stabilizing, tube-based, model predictive controller. The state estimation error is bounded by an invariant set. The tube-based controller ensures that all possible realizations of the state trajectory lie in a simple uncertainty tube the ‘center’ of which is the solution of a nominal (disturbance-free) system and the ‘cross-section’ of which is also invariant. Satisfaction of the state and input constraints for the original system is guaranteed by employing tighter constraint sets for the nominal system. The complexity of the resultant controller is similar to that required for nominal model predictive control.     

Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games

In this paper we consider a finite horizon, nonlinear, stochastic, risk-sensitive optimal control problem with complete state information, and show that it is equivalent to a stochastic differential game. Risk-sensitivity and small noise parameters are introduced, and the limits are analyzed as these parameters tend to zero. First-order expansions are obtained which show that the risk-sensitive controller consists of a standard deterministic controller, plus terms due to stochastic and game-theoretic methods of controller design. The results of this paper relate to the design of robust controllers for nonlinear systems.Research supported in part by the 1990 Summer Faculty Research Fellowship, University of Kentucky.     

General finite-dimensional risk-sensitive problems and small noiselimits

For a risk-sensitive, partially observed stochastic control problem, the modified Zakai equation includes an extra term related to the exponential running cost. The finite-dimensional solutions of this modified Zakai equation are obtained. These are analogs of the Kalman and Benes filters. The small noise limits of the finite-dimensional risk-sensitive problems are then obtained. These lead to differential games with deterministic disturbances     

严格反馈非线性系统预设性能backstepping 控制器设计

针对一类具有一般形式的严格反馈非线性系统,提出一种基于预设性能的backstepping控制器设计方法.所谓预设性能是指在保证跟踪误差收敛到一个预先设定的任意小的区域的同时,保证收敛速度及超调量满足预先设定的条件.首先引入性能函数的概念,通过误差转化将原始的受限系统转换为等价的非受限系统;然后基于Lyapunov理论进行backstepping控制器的设计,并进行了稳定性分析;最后通过仿真实验验证了所提出方法的正确性.     

The role of information state and adjoint in relating nonlinearoutput feedback risk-sensitive control and dynamic games

This paper employs logarithmic transformations to establish relations between continuous-time nonlinear partially observable risk-sensitive control problems and analogous output feedback dynamic games. The first logarithmic transformation is introduced to relate the stochastic information state to a deterministic information state. The second logarithmic transformation is applied to the risk-sensitive cost function using the Laplace-Varadhan lemma. In the small noise limit, this cost function is shown to be logarithmically equivalent to the cost function of an analogous dynamic game     

Stochastic nonlinear minimax dynamic games with noisy measurements

This note is concerned with nonlinear stochastic minimax dynamic games which are subject to noisy measurements. The minimizing players are control inputs while the maximizing players are square-integrable stochastic processes. The minimax dynamic game is formulated using an information state, which depends on the paths of the observed processes. The information state satisfies a partial differential equation of the Hamilton-Jacobi-Bellman (HJB) type. The HJB equation is employed to characterize the dissipation properties of the system, to derive a separation theorem between the design of the estimator and the controller, and to introduce a certainty-equivalence principle along the lines of Whittle. Finally, the separation theorem and the certainty-equi. valence principle are applied to solve the linear-quadratic-Gaussian minimax game. The results of this note generalize the L/sup 2/-gain of deterministic systems to stochastic analogs; they are related to the controller design of stochastic systems which employ risk-sensitive performance criteria, and to the controller design of deterministic systems which employ minimax performance criteria.     

Average cost criterion induced by the regular utility function for continuous-time Markov decision processes

In this paper we study the average cost criterion induced by the regular utility function (U-average cost criterion) for continuous-time Markov decision processes. This criterion is a generalization of the risk-sensitive average cost and expected average cost criteria. We first introduce an auxiliary risk-sensitive first passage optimization problem and obtain the properties of the corresponding optimal value function under the slight conditions. Then we show that the pair of the optimal value functions of the risk-sensitive average cost criterion and the risk-sensitive first passage criterion is a solution to the optimality equation of the risk-sensitive average cost criterion allowing the risk-sensitivity parameter to take any nonzero value. Moreover, we have that the optimal value function of the risk-sensitive average cost criterion is continuous with respect to the risk-sensitivity parameter. Finally, we give the connections between the U-average cost criterion and the average cost criteria induced by the identity function and the exponential utility function, and prove the existence of a U-average optimal deterministic stationary policy in the class of all randomized Markov policies.     

Connections between stochastic control and dynamic games

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous- and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.     

World of Warcraft and the impact of game culture and play in an undergraduate game design course

During the past two decades, digital games have become an increasingly popular source of study for academics, educational researchers and instructional designers. Much has been written about the potential of games for teaching and learning, both in the design of educational/serious games and the implementation of off-the-shelf games for learning. Yet relatively little research has been conducted about how game culture and the enmeshed practice of play may impact classroom dynamics. The purpose of this study is to present a case study about how the use of World of Warcraft (WoW) as a teaching tool and medium of play impacted class dynamics in an undergraduate university-level course for game design. Specifically, this study will address how WoW’s game culture and the practice of play impacted (a) student-to-student dynamics and (b) class dynamics. The goal of this study is to explore some of the dynamics of play as a component of learning.     

Flipping the winner of a poset game

Partially-ordered set games, also called poset games, are a class of two-player combinatorial games. The playing field consists of a set of elements, some of which are greater than other elements. Two players take turns removing an element and all elements greater than it, and whoever takes the last element wins. Examples of poset games include Nim and Chomp. We investigate the complexity of computing which player of a poset game has a winning strategy. We give an inductive procedure that modifies poset games to change the nim-value which informally captures the winning strategies in the game. For a generic poset game G, we describe an efficient method for constructing a game ¬G such that the first player has a winning strategy if and only if the second player has a winning strategy on G. This solves the long-standing problem of whether this construction can be done efficiently. This construction also allows us to reduce the class of Boolean formulas to poset games, establishing a lower bound on the complexity of poset games.     

Risk-sensitive dual control

In this paper, we develop new results concerning the risk-sensitive dual control problem for output feedback nonlinear systems, with unknown time-varying parameters. These results are not merely immediate specializations of known risk-sensitive control theory for nonlinear systems, but rather, are new formulations which are of interest in their own right. A dynamic programming equation solution is given to an optimal risk-sensitive dual control problem penalizing outputs, rather than the states, for a reasonably general class of nonlinear signal models. This equation, in contrast to earlier formulations in the literature, clearly shows the dual aspects of the risk-sensitive controller regarding control and estimation. The computational task to solve this equation, as has been seen for the risk-neutral dual control problem, suffers from the so-called ‘curse of dimensionality’. This motivates our study of the risk-sensitive version for a suboptimal risk-sensitive dual controller. Explicit controllers are derived for a minimum phase single-input, single-output auto-regressive model with exogenous input and unknown time-varying parameters. Also, simulation studies are carried out for an integrator with a time-varying gain. They show that the risk-sensitive suboptimal dual controller is more robust to uncertain noise environments compared with its risk-neutral counterpart. © 1997 by John Wiley & Sons, Ltd.     

Fuzzy Optimal Control for Bilinear Stochastic Systems with Fuzzy Parameters

In this paper we consider the control problem for a class of partially observed bilinear stochastic systems with fuzzy parameters. Using Takagi–Sugeno fuzzy model, the problem is described by three sets of fuzzy stochastic differential equations: one for the state process, one for the observed process and one for the controller which is assumed to be driven by the observed process. With this formulation, the original stochastic control problem can be treated as a deterministic identification problem in which the controller parameters and the corresponding membership functions are the unknowns. Using a suitable performance index, we have developed a set of necessary conditions for determining the parameters of the controller and the corresponding membership functions. Finally, some numerical simulations are presented to illustrate the effectiveness of the proposed fuzzy control scheme.     

A game theoretic approach to the optimal control of uncertain systems

The issue of performance robustness is considered in a dynamic games framework where the goal is to find a feedback controller that stabilizes the system under all plant perturbations and that minimizes the value of a quadratic performance index under the worst perturbations. To find a solution to this hard-constrained problem, an auxiliary soft-constrained dynamic game problem is formulated. It is shown that the solution to the auxiliary problem provides the solution to the original problem, under the assumption that the solution to the original problem occurs on the boundary of the perturbations set. The approach, however, can still be applied, with modifications, to the case when this assumption fails.