Q‐learning for continuous‐time graphical games on large networks with completely unknown linear system dynamics

In this paper, we consider the problem of leader synchronization in systems with interacting agents in large networks while simultaneously satisfying energy‐related user‐defined distributed optimization criteria. But modeling in large networks is very difficult, and for that reason, we derive a model‐free formulation that is based on a separate distributed Q‐learning function for every agent. Every Q‐function is a parametrization of each agent's control, of the neighborhood controls, and of the neighborhood tracking error. It is also evident that none of the agents has any information on where the leader is connected to and from where she spreads the desired information. The proposed algorithm uses an integral reinforcement learning approach with a separate distributed actor/critic network for each agent: a critic approximator to approximate each value function and an actor approximator to approximate each optimal control law. The derived tuning laws for each actor and critic approximators are designed appropriately by using gradient descent laws. We provide rigorous stability and convergence proofs to show that the closed‐loop system has an asymptotically stable equilibrium point and that the control policies form a graphical Nash equilibrium. We demonstrate the effectiveness of the proposed method on a network consisting of 10 agents. Copyright © 2016 John Wiley & Sons, Ltd.     

Event‐based global stabilization of linear systems via a saturated linear controller

This paper investigates the problem of event‐based linear control of systems subject to input saturation. First, for discrete‐time systems with neutrally stable or double‐integrator dynamics, novel event‐triggered control algorithms with non‐quadratic event‐triggering conditions are proposed to achieve global stabilization. Compared with the quadratic event‐triggering conditions, the non‐quadratic ones can further reduce unnecessary control updates for the input‐saturated systems. Furthermore, for continuous‐time systems with neutrally stable or double‐integrator dynamics, because an inherent lower bound of the inter‐event time does not exist for systems subject to input saturation, novel event‐triggered control algorithms with an appropriately selected minimum inter‐event time are proposed to achieve global stabilization. Finally, numerical examples are provided to illustrate the theoretical results. Copyright © 2015 John Wiley & Sons, Ltd.     

An iterative Q‐learning scheme for the global stabilization of discrete‐time linear systems subject to actuator saturation

In this paper, we propose a model‐free algorithm for global stabilization of linear systems subject to actuator saturation. The idea of gain‐scheduled low gain feedback is applied to develop control laws that avoid saturation and achieve global stabilization. To design these control laws, we employ the framework of parameterized algebraic Riccati equations (AREs). Reinforcement learning techniques are developed to find the solution of the parameterized ARE without requiring any knowledge of the system dynamics. In particular, we present an iterative Q‐learning scheme that searches for a low gain parameter and iteratively solves the parameterized ARE using the Bellman equation. Both state feedback and output feedback algorithms are developed. It is shown that the proposed scheme achieves model‐free global stabilization under bounded controls and convergence to the optimal solution of the ARE is achieved. Simulation results are presented that confirm the effectiveness of the proposed method.     

A Lyapunov‐based small‐gain approach on design of triggering conditions in event‐triggered control systems

This paper studies a Lyapunov‐based small‐gain approach on design of triggering conditions in event‐triggered control systems. The event‐triggered control closed‐loop system is formulated as a hybrid system model. Firstly, by viewing the event‐triggered control closed‐loop system as a feedback connection of two smaller hybrid subsystems, the Lyapunov‐based small‐gain theorems for hybrid systems are applied to design triggering conditions. Then, a new class of triggering condition, the safe, adjustable‐type triggering condition, is proposed to tune the parameters of triggering conditions by practical regulations. This is conducive to break the restriction of the conservation of theoretical results and improve the practicability of event‐triggered control strategy. Finally, a numerical example is given to illustrate the efficiency and the feasibility of the proposed results. Copyright © 2015 John Wiley & Sons, Ltd.     

Reducing correlation of random forest–based learning‐to‐rank algorithms using subsample size

Learning‐to‐rank (LtR) has become an integral part of modern ranking systems. In this field, the random forest–based rank‐learning algorithms are shown to be among of the top performers. Traditionally, each tree of a random forest is learnt using a bootstrapped copy of the training set, where approximately 63% of the examples are unique. The goal of using a bootstrapped copy instead of the original training set is to reduce the correlation between individual trees, thereby making the prediction of the ensemble more accurate. In this regard, the following question may be raised: how can we leverage the correlation between the trees in favor of performance and scalability of a random forest–based LtR algorithm? In this article, we investigate whether we can further decrease the correlation between the trees while maintaining or possibly improving accuracy. Among several potential options to achieve this goal, we investigate the size of the subsamples used for learning individual trees. We examine the performance of a random forest–based LtR algorithm as we control the correlation using this parameter. Experiments on LtR data sets reveal that for small‐ to moderate‐sized data sets, substantial reduction in training time can be achieved using only a small amount of training data per tree. Moreover, due to the positive correlation between the variability across the trees and performance of a random forest, we observe an increase in accuracy while maintaining the same level of model stability as the baseline. For big data sets, although our experiments did not observe an increase in accuracy (because, with larger data sets, the individual tree variance is already comparatively smaller), our technique is still applicable as it allows for greater scalability.     

Formation control for a class of nonlinear multiagent systems using model‐free adaptive iterative learning

In this paper, the problem of formation control is considered for a class of unknown nonaffine nonlinear multiagent systems under a repeatable operation environment. To achieve the formation objective, the unknown nonlinear agent's dynamic is first transformed into a compact form dynamic linearization model along the iteration axis. Then, a distributed model‐free adaptive iterative learning control scheme is designed to ensure that all agents can keep their desired deviations from the reference trajectory over the whole time interval. The main results are given for the multiagent systems with fixed communication topologies and the extension to the switching topologies case is also discussed. The feature of this design is that formation control can be solved only depending on the input/output data of each agent. An example is given to demonstrate the effectiveness of the proposed method.     

Dynamic output‐feedback fuzzy MPC for Takagi‐Sugeno fuzzy systems under event‐triggering–based try‐once‐discard protocol

The fuzzy model predictive control (FMPC) problem is studied for a class of discrete‐time Takagi‐Sugeno (T‐S) fuzzy systems with hard constraints. In order to improve the network utilization as well as reduce the transmission burden and avoid data collisions, a novel event‐triggering–based try‐once‐discard (TOD) protocol is developed for networks between sensors and the controller. Moreover, due to practical difficulties in obtaining measurements, the dynamic output‐feedback method is introduced to replace the traditional state feedback method for addressing the FMPC problem. Our aim is to design a series of controllers in the framework of dynamic output‐feedback FMPC for T‐S fuzzy systems so as to find a good balance between the system performance and the time efficiency. Considering nonlinearities in the context of the T‐S fuzzy model, a “min‐max” strategy is put forward to formulate an online optimization problem over the infinite‐time horizon. Then, in light of the Lyapunov‐like function approach that fully involves the properties of the T‐S fuzzy model and the proposed protocol, sufficient conditions are derived to guarantee the input‐to‐state stability of the underlying system. In order to handle the side effects of the proposed event‐triggering–based TOD protocol, its impacts are fully taken into consideration by virtue of the S‐procedure technique and the quadratic boundedness methodology. Furthermore, a certain upper bound of the objective is provided to construct an auxiliary online problem for the solvability, and the corresponding algorithm is given to find the desired controllers. Finally, two numerical examples are used to demonstrate the validity of proposed methods.     

Formal methods applied to high‐performance computing software design: a case study of MPI one‐sided communication‐based locking

There is a growing need to address the complexity of verifying the numerous concurrent protocols employed in the high‐performance computing software. Today's approaches for verification consist of testing detailed implementations of these protocols. Unfortunately, this approach can seldom show the absence of bugs, and often results in serious bugs escaping into the deployed software. An approach called Model Checking has been demonstrated to be eminently helpful in debugging these protocols early in the software life cycle by offering the ability to represent and exhaustively analyze simplified formal protocol models. The effectiveness of model checking has yet to be adequately demonstrated in high‐performance computing. This paper presents a case study of a concurrent protocol that was thought to be sufficiently well tested, but proved to contain two very non‐obvious deadlocks in them. These bugs were automatically detected through model checking. The protocol models in which these bugs were detected were also easy to create. Recent work in our group demonstrates that even this tedium of model creation can be eliminated by employing dynamic source‐code‐level analysis methods. Our case study comes from the important domain of Message Passing Interface (MPI)‐based programming, which is universally employed for simulating and predicting anything from the structural integrity of combustion chambers to the path of hurricanes. We argue that model checking must be taught as well as used widely within HPC, given this and similar success stories. Copyright © 2009 John Wiley & Sons, Ltd.     

Robust observer‐based stabilization of Lipschitz nonlinear uncertain systems via LMIs ‐ discussions and new design procedure

This paper presents a new observer‐based controller design method for Lipschitz nonlinear systems with uncertain parameters and ‐bounded disturbance inputs. In the presence of uncertain parameters, the separation principle is not applicable even in the case of linear time invariant systems. A state of the art review for uncertain linear systems is first presented to describe the shortcomings and conservatism of existing results for this problem. Then a new LMI‐based design technique is developed to solve the problem for both linear and Lipschitz nonlinear systems. The features of the new technique are the use of a new matrix decomposition, the allowance of additional degrees of freedom in design of the observer and controller feedback gains, the elimination of any need to use equality constraints, the allowance of uncertainty in the input matrix and the encompassing of all previous results under one framework. An extensive portfolio of numerical case studies is presented to illustrate the superiority of the developed design technique to existing results for linear systems from literature and to illustrate application to Lipschitz nonlinear systems. Copyright © 2016 John Wiley & Sons, Ltd.