Towards to dynamic optimal control for large-scale distributed systems

There is a class of complex, large-scale systems that are composed of many interacted and spatially distributed subsystems such as the power grids, transportation systems, and large scale chemical processes. With the progress of industrial technology, field bus, integrated circuit technologies and network communication technology, this automation system has taken a totally new configuration in which each subsystem owns only one hardware controller for controlling the subsystem itself, and all hardware controllers are connected by networks for exchanging information with each other or with upper layer controllers. In this distributed (or decentralized) framework, the distributed control algorithms are usually adopted because the classical centralized control solutions are often impractical due to their heavy computational demands and the lack of fault tolerance....     

An object-oriented approach to the management of distributed application systems

General purpose distributed object-oriented environments exist to allow for the efficient construction of client/server software systems. Standard network and distributed systems management environments exist for the efficient operation of heterogeneous networked hardware and software systems. As distributed software systems get larger, the economies of systems development and the economies of software operation demand that we find an efficient way of integrating these two technologies. While the use of standardized distributed systems management for the management of distributed software applications seems reasonable, very little research has been done to confirm this. In this paper, we propose the integration of standardized distributed resource management technologies and distributed application software. In our work we have facilitated this integration using techniques based on mainstream object-oriented dynamic modeling. We will describe our techniques of integration as well as discuss the need for methodical engineered approaches when working in this area.     

Dynamic balancing of communication and computation load for HLA-based simulations on large-scale distributed systems

Dynamic balancing of computation and communication load is vital for the execution stability and performance of distributed, parallel simulations deployed on the shared, unreliable resources of large-scale environments. High Level Architecture (HLA) based simulations can experience a decrease in performance due to imbalances that are produced initially and/or during run time. These imbalances are generated by the dynamic load changes of distributed simulations or by unknown, non-managed background processes resulting from the non-dedication of shared resources. Due to the dynamic execution characteristics of elements that compose distributed applications, the computational load and interaction dependencies of each simulation entity change during run time. These dynamic changes lead to an irregular load and communication distribution, which increases overhead of resources and latencies. A static partitioning of load is limited to deterministic applications and is incapable of predicting the dynamic changes caused by distributed applications or by external background processes. Therefore, a scheme for balancing the communication and computational load during the execution of distributed simulations is devised in a scalable hierarchical architecture. The proposed balancing system employs local and cluster monitoring mechanisms in order to observe the distributed load changes and identify imbalances, repartitioning policies to determine a distribution of load and minimize imbalances. A migration technique is also employed by this proposed balancing system to perform reliable and low-latency load transfers. Such a system successfully improves the use of shared resources and increases distributed simulations’ performance by minimizing communication latencies and partitioning the load evenly. Experiments and comparative analyses were conducted in order to identify the gains that the proposed balancing scheme provides to large-scale distributed simulations.     

Lotka-Volterra-like approach to large-scale systems stability

A new aggregation form of large-scale systems is proposed in this paper. It is of the ‘Lotka-Volterra’ type and appears natural for the generalized Lotka-Volterra large-scale (possibly time-varying) systems, which is shown in the paper. The Lotka-Volterra aggregation form yields a comparison system of the generalized Lotka-Volterra type, which is composed of a linear and a non-linear part. In addition this aggregation form leads to new stability conditions for time-varying large-scale systems, and enables an extension of Weissenberger's results on exponential stability domain estimates to the asymptotic stability domain estimate of time-varying systems. An advantageous feature of the system aggregation and stability analysis developed in the paper is the omission of requirements for (any particular, e.g. exponential) stability of disconnected subsystems.     

A two-phase approach to interactivity enhancement for large-scale distributed virtual environments

Distributed virtual environments (DVEs) are distributed systems that allow multiple geographically distributed clients (users) to interact simultaneously in a computer-generated, shared virtual world. Applications of DVEs can be seen in many areas nowadays, such as online games, military simulations, collaborative designs, etc. To support large-scale DVEs with real-time interactions among thousands or even more distributed clients, a geographically distributed server architecture (GDSA) is generally needed, and the virtual world can be partitioned into many distinct zones to distribute the load among the servers. Due to the geographic distributions of clients and servers in such architectures, it is essential to efficiently assign the participating clients to servers to enhance users’ experience in interacting within the DVE. This problem is termed the client assignment problem (CAP) in this paper. We propose a two-phase approach, consisting of an initial assignment phase and a refined assignment phase to address the CAP. Both phases are shown to be NP-hard. Several heuristic assignment algorithms are then devised and evaluated via extensive simulations with realistic settings. We find that, even under heterogeneous environments like the Internet where accurate input data for the assignment algorithms are usually impractical to obtain, the proposed algorithms are still beneficial to the performances of DVE.     

Dynamic distributed systems

This paper introduces a new methodology for building flexible and programmable multiprocessor systems. It is based on a distributed implementation of monitors which is supported by a system-wide address relocation mechanism. This provides the dynamic reallocation of processes and monitors within the system. Methods of implementing the communications subsystem based on a number of common network architectures are presented. An algorithm which effects dynamic resource allocation is presented. Its effect is that groups of processes which are interacting through a monitor will tend to accumulate at various sites in the system in a manner reflecting the dynamic interaction patterns of the program. This is accomplished without there being any centralized control, or even knowledge of the overall program interactions. The feasibility of the system has been investigated using a model system running on a loosely connected dual processor system.     

Independent recovery in large-scale distributed systems

In large systems, replication can become important means to improve data access times and availability. Existing recovery protocols, on the other hand, were proposed for small-scale distributed systems. Such protocols typically update stale, newly-recovered sites with replicated data and resolve the commit uncertainty of recovering sites. Thus, given that in large systems failures are more frequent and that data access times are costlier, such protocols can potentially introduce large overheads in large systems and must be avoided, if possible. We call these protocols dependent recovery protocols since they require a recovering site to consult with other sites. Independent recovery has been studied in the context of one-copy systems and has been proven unattainable. This paper offers independent recovery protocols for large-scale systems with replicated data. It shows how the protocols can be incorporated into several well-known replication protocols and proves that these protocols continue to ensure data consistency. The paper then addresses the issue of nonblocking atomic commitment. It presents mechanisms which can reduce the overhead of termination protocols and the probability of blocking. Finally, the performance impact of the proposed recovery protocols is studied through the use of simulation and analytical studies. The results of these studies show that the significant benefits of independent recovery can be enjoyed with a very small loss in data availability and a very small increase in the number of transaction abortions     

Object-based approach to programming distributed systems

Two kinds of parallel computers exist: those with shared memory and those without. The former are difficult to build but easy to program. The latter are easy to build but difficult to program. In this paper we present a hybrid model that combines the best properties of each by simulating a restricted object-based shared memory on machines that do not share physical memory. In this model, objects can be replicated on multiple machines. An operation that does not change an object can then be done locally, without any network traffic. Update operations can be done using the reliable broadcast protocol described in the paper. We have constructed a prototype system, designed and implemented a new programming language for it, and programmed various applications using it. The model, algorithms, language, applications and performance will be discussed.     

Adaptive diagnosis in distributed systems

Real-time problem diagnosis in large distributed computer systems and networks is a challenging task that requires fast and accurate inferences from potentially huge data volumes. In this paper, we propose a cost-efficient, adaptive diagnostic technique called active probing . Probes are end-to-end test transactions that collect information about the performance of a distributed system. Active probing uses probabilistic reasoning techniques combined with information-theoretic approach, and allows a fast online inference about the current system state via active selection of only a small number of most-informative tests. We demonstrate empirically that the active probing scheme greatly reduces both the number of probes (from 60% to 75% in most of our real-life applications), and the time needed for localizing the problem when compared with nonadaptive (preplanned) probing schemes. We also provide some theoretical results on the complexity of probe selection, and the effect of "noisy" probes on the accuracy of diagnosis. Finally, we discuss how to model the system's dynamics using dynamic Bayesian networks (DBNs), and an efficient approximate approach called sequential multifault; empirical results demonstrate clear advantage of such approaches over "static" techniques that do not handle system's changes.     

Adaptive event-triggered state estimation for large-scale systems subject to deception attacks

This paper addresses state estimation issues of large-scale systems with measurements subject to deception attacks,where the communication topology among sub-es...     

Nash-based robust distributed model predictive control for large-scale systems

In this paper, a new robust distributed model predictive control (RDMPC) is proposed for large-scale systems with polytopic uncertainties. The time-varying system is first decomposed into several interconnected subsystems. Interactions between subsystems are obtained by a distributed Kalman filter, in which unknown parameters of the system are estimated using local measurements and measurements of neighboring subsystems that are available via a network. Quadratic boundedness is used to guarantee the stability of the closed-loop system. In the MPC algorithm, an output feedback-interaction feedforward control input is computed by an LMI-based optimization problem that minimizes an upper bound on the worst case value of an infinite-horizon objective function. Then, an iterative Nash-based algorithm is presented to achieve the overall optimal solution of the whole system in partially distributed fashion. Finally, the proposed distributed MPC approach is applied to a load frequency control (LFC) problem of a multi-area power network to study the efficiency and applicability of the algorithm in comparison with the centralized, distributed and decentralized MPC schemes.     

不确定关联大系统对时变参数的自适应控制

考虑具有时滞的不确定非线性关联大系统的鲁棒控制问题．假设不确定时变参数为半线性或非线性系统的有界输出，通过对时变不确定参数设计自适应律，从而对不确定参数进行估计．利用线性矩阵不等式技术和自适应参数估计方法，设计出鲁棒自适应控制器，从而保证闭环系统渐近稳定．建立了可由线性矩阵不等式表示的镇定条件．仿真示例说明该方法是有效的．     

Adaptive rational Krylov subspaces for large-scale dynamical systems

The rational Krylov space is recognized as a powerful tool within model order reduction techniques for linear dynamical systems. However, its success has been hindered by the lack of a parameter-free procedure, which would effectively generate the sequence of shifts used to build the space. In this paper we propose an adaptive computation of these shifts. The whole procedure only requires us to inject some initial rough estimate of the spectral region of the matrix, while further information is automatically generated during the process. The approach is a full generalization to the nonsymmetric case of the idea first proposed in Druskin et al. (2010) [18] and it is used for two important problems in control: the approximation of the transfer function and the numerical solution of large Lyapunov equations. The procedure can be naturally extended to other related problems, such as the solution of the Sylvester equation, and parametric or higher order systems. Several numerical experiments are proposed to assess the quality of the rational projection space over its most natural competitors.     

A codesign approach for distributed systems

Distributed systems have become prevalent in response to the rapidly expanding Internet's demands. Their design presents new challenges because it involves the interaction of hardware and software. Continual marketplace innovation drives computing toward heterogeneity in both hardware and software and generates a complexity that goes beyond the earlier codesign approaches, which were developed for more homogeneous systems executing in non-distributed environments. Codesign of heterogeneous systems requires the support of a powerful modeling and simulation environment because analysis alone cannot deal with all the challenges such complex systems pose. We believe that modeling and simulation, using the discrete-event system specification modeling and simulation framework, are the most suitable vehicles to study the complexities associated with developing distributed-object computing systems     

Risk modeling in distributed, large-scale systems

Risk is inherent in distributed, large-scale systems. The paper explores the challenges of risk modeling in such systems, and suggests a risk modeling approach that is responsive to the requirements of complex, distributed, large-scale systems. An example of the use of the approach in the marine transportation system is given. The paper concludes with a discussion of limitations of the approach and of future work     

Dynamic Grid-Based Approach to Data Distribution Management

Data distribution management (DDM) is one of the services defined by the DoD High Level Architecture and is necessary to provide efficient, scalable mechanisms for distributing state updates and interaction information in large scale distributed simulations. In this paper, we focus on data distribution management mechanisms (also known as filtering) used for real time training simulations. We propose a new method of DDM, which we refer to as the dynamic grid-based approach. Our scheme is based on a combination of a fixed grid-based method, known for its scalability, and a region-based strategy, which provides greater accuracy than the fixed grid-based method. We describe our DDM algorithm, its implementation, and report on the performance results that we have obtained using the RTI-Kit framework. Our results clearly indicate that our scheme is scalable and that it reduces the message overhead by 40%, and the number of multicast groups used by 98% when compared to the fixed grid-based allocation scheme using 10 nodes, 1000 objects, and 20,000 grid cells.     

An approach to efficient distributed transactions

Most distributed systems proposed on the basis of the concept of atomic action or transaction strongly limit parallelism, thus reducing their level of efficiency. In this paper, features of efficiency in a distributed transaction system are investigated. Two mechanisms are proposed in order to enhance potential concurrency both among different transactions and within a single transaction during the commit phase:

- a synchronization mechanism has been designed which suggests an approach to concurrency control by allowing the release of acquired locks before transaction completion. The possibility of exploiting this mechanism to implement nested transactions is also discussed.

- a distributed commit protocol is developed which enhances concurrency among the participants in an atomic action, thus achieving quick execution with high modularity.

     

Data-driven model reduction-based nonlinear MPC for large-scale distributed parameter systems

Model predictive control (MPC) has been effectively applied in process industries since the 1990s. Models in the form of closed equation sets are normally needed for MPC, but it is often difficult to obtain such formulations for large nonlinear systems. To extend nonlinear MPC (NMPC) application to nonlinear distributed parameter systems (DPS) with unknown dynamics, a data-driven model reduction-based approach is followed. The proper orthogonal decomposition (POD) method is first applied off-line to compute a set of basis functions. Then a series of artificial neural networks (ANNs) are trained to effectively compute POD time coefficients. NMPC, using sequential quadratic programming is then applied. The novelty of our methodology lies in the application of POD's highly efficient linear decomposition for the consequent conversion of any distributed multi-dimensional space-state model to a reduced 1-dimensional model, dependent only on time, which can be handled effectively as a black-box through ANNs. Hence we construct a paradigm, which allows the application of NMPC to complex nonlinear high-dimensional systems, even input/output systems, handled by black-box solvers, with significant computational efficiency. This paradigm combines elements of gain scheduling, NMPC, model reduction and ANN for effective control of nonlinear DPS. The stabilization/destabilization of a tubular reactor with recycle is used as an illustrative example to demonstrate the efficiency of our methodology. Case studies with inequality constraints are also presented.