Convergence of gradient projection routing methods in anasynchronous stochastic quasi-static virtual circuit network

The convergence of the gradient projection algorithms for optimal routing in virtual circuit data networks proposed by D.P. Bertsekas (1982) is studied. The routing model explicitly takes into account stochastic generation and termination of virtual circuits, distributed asynchronous routing updates, inaccurate flow measurement, and delays in forwarding control packets. The problem of assigning paths for incoming sessions (or virtual circuits) to implement the gradient projection algorithms is also studied. A metering rule based on deficiency in a desired number of virtual circuits is proposed and analyzed. It is shown that the proposed metering rule is better than a randomized rule in some sense. The gradient projection routing algorithms implemented either by the metering rule or the randomized rule are shown to converge to a neighborhood of a long-term optimal routing     

A shortest path routing algorithm using Hopfield neural network with an improved energy function

A shortest path routing algorithm using the Hopfield neural network with a modified Lyapunov function is proposed. The modified version of the Lyapunov energy function for an optimal routing problem is proposed for determining routing order for a source and multiple destinations. The proposed energy function mainly prevents the solution path from having loops and partitions. Experiments are performed on 3000 networks of up to 50 nodes with randomly selected link costs. The performance of the proposed algorithm is compared with several conventional algorithms including Ali and Kamoun's, Park and Choi's, and Ahn and Ramakrishna's algorithms in terms of the route optimality and convergence rate. The results show that the proposed algorithm outperforms conventional methods in all cases of experiments. The proposed algorithm particularly shows significant improvements on the route optimality and convergence rate over conventional algorithms when the size of the network approaches 50 nodes.     

一类混杂系统的最优控制

研究了一类脉冲依赖于状态的混杂系统的最优控制问题.与传统的变分方法不同,通过将跳跃瞬间转化为一个新的待优化参数,得到了该混杂系统的必要最优性条件,从而将最优控制问题转化为一边界值问题,该边界值问题可由数值方法或解析方法解决.此外,利用广义微分的理论,将该必要最优性条件推广到Frechet微分形式.结论表明,在混杂动态系统运行的连续部分,最优解所满足的必要性条件和传统的连续系统相同.在混杂动态系统的脉冲点处,哈密尔顿函数满足连续性条件,协态变量则满足一定的跳跃条件.最后,通过两个实例分析,表明该方法是有效的.     

An optimal regulation strategy with disturbance rejection for energy management of hybrid electric vehicles

The energy management problem of finding the optimal split between the different sources of energy in a charge-sustaining parallel HEV, ensuring stability and optimality with respect to a performance objective (fuel consumption minimization over a driving cycle), is addressed in this paper. The paper develops a generic stability and optimality framework within which the energy management problem is cast in the form of a nonlinear optimal regulation (with disturbance rejection) problem and a control Lyapunov function is used to design the control law. Two theorems ensuring optimality and asymptotic stability of the energy management strategy are proposed and proved. The sufficient conditions for optimality and stability are used to derive an analytical expression for the control law as a function of the battery state of charge/state of energy and system parameters. The control law is implemented in a simplified backward vehicle simulator and its performance is evaluated against the global optimal solution obtained from dynamic programming. The strategy performs within 4% of the benchmark solution while guaranteeing optimality and stability for any driving cycle.     

An optimal control approach to dynamic routing in networks

This paper explores the application of optimal control theory to the problem of dynamic routing in networks. The approach derives from a continuous state space model for dynamic routing and an associated linear optimal control problem with linear state and control variable inequality constraints. The conceptual form of an algorithm is presented for finding a feedback solution to the optimal control problem when the inputs are assumed to be constant in time. The algorithm employs a combination of necessary conditions, dynamic programming, and linear programming to construct a set of convex polyhedral cones which cover the admissible state space with optimal controls. An implementable form of the algorithm, along with a simple example, is presented for a special class of single destination networks.     

Optimal control for multi-stage and continuous-time linear singular systems

In this paper, optimal control problems for multi-stage and continuous-time linear singular systems are both considered. The singular systems are assumed to be regular and impulse-free. First, a recurrence equation is derived according to Bellman's principle of optimality in dynamic programming. Then, by applying the recurrence equation, bang-bang optimal controls for the control problems with linear objective functions subject to two types of multi-stage singular systems are obtained. Second, employing the principle of optimality, a equation of optimality for settling the optimal control problem subject to a class of continuous-time singular systems is proposed. The optimal control problem may become simpler through solving this equation of optimality. Two numerical examples and a dynamic input–output model are presented to show the effectiveness of the results obtained.     

Infinite horizon optimal control of affine nonlinear discrete switched systems using two-stage approximate dynamic programming

In this article, a novel iteration algorithm named two-stage approximate dynamic programming (TSADP) is proposed to seek the solution of nonlinear switched optimal control problem. At each iteration of TSADP, a multivariate optimal control problem is transformed to be a certain number of univariate optimal control problems. It is shown that the value function at each iteration can be characterised pointwisely by a set of smooth functions recursively obtained from TSADP, and the associated control policy, continuous control and switching control law included, is explicitly provided in a state-feedback form. Moreover, the convergence and optimality of TSADP is strictly proven. To implement this algorithm efficiently, neural networks, critic and action networks, are utilised to approximate the value function and continuous control law, respectively. Thus, the value function is expressed by the weights of critic networks pointwise. Besides, redundant weights are ruled out at each iteration to simplify the exponentially increasing computation burden. Finally, a simulation example is provided to demonstrate its effectiveness.     

Lipschitzian Regularity of the Minimizing Trajectories for Nonlinear Optimal Control Problems

We consider the Lagrange problem of optimal control with unrestricted controls and address the question: under what conditions can we assure optimal controls are bounded? This question is related to one of Lipschitzian regularity of optimal trajectories, and the answer to it is crucial in closing the gap between the conditions arising in existence theory and necessary optimality conditions. Rewriting the Lagrange problem in a parametric form, we obtain a relation between the applicability conditions of the Pontryagin maximum principle to the latter problem and the Lipschitzian regularity conditions for the original problem. Under the standard hypotheses of coercivity of the existence theory, the conditions imply that the optimal controls are essentially bounded, assuring the applicability of the classical necessary optimality conditions like the Pontryagin maximum principle. The result extends previous Lipschitzian regularity results to cover optimal control problems with general nonlinear dynamics.     

Optimality of affine control system of several species in competition on a sequential batch reactor

In this paper, we analyse the optimality of affine control system of several species in competition for a single substrate on a sequential batch reactor, with the objective being to reach a given (low) level of the substrate. We allow controls to be bounded measurable functions of time plus possible impulses. A suitable modification of the dynamics leads to a slightly different optimal control problem, without impulsive controls, for which we apply different optimality conditions derived from Pontryagin principle and the Hamilton–Jacobi–Bellman equation. We thus characterise the singular trajectories of our problem as the extremal trajectories keeping the substrate at a constant level. We also establish conditions for which an immediate one impulse (IOI) strategy is optimal. Some numerical experiences are then included in order to illustrate our study and show that those conditions are also necessary to ensure the optimality of the IOI strategy.     

Data‐driven optimal event‐triggered consensus control for unknown nonlinear multiagent systems with control constraints

This paper considers optimal consensus control problem for unknown nonlinear multiagent systems (MASs) subjected to control constraints by utilizing event‐triggered adaptive dynamic programming (ETADP) technique. To deal with the control constraints, we introduce nonquadratic energy consumption functions into performance indices and formulate the Hamilton‐Jacobi‐Bellman (HJB) equations. Then, based on the Bellman's optimality principle, constrained optimal consensus control policies are designed from the HJB equations. In order to implement the ETADP algorithm, the critic networks and action networks are developed to approximate the value functions and consensus control policies respectively based on the measurable system data. Under the event‐triggered control framework, the weights of the critic networks and action networks are only updated at the triggering instants which are decided by the designed adaptive triggered conditions. The Lyapunov method is used to prove that the local neighbor consensus errors and the weight estimation errors of the critic networks and action networks are ultimately bounded. Finally, a numerical example is provided to show the effectiveness of the proposed ETADP method.     

Optimal control of two interacting service stations

Optimal controls described by switching curves in the two-dimensional state space are shown to exist for the optimal control of a Markov network with two service stations and linear cost. The controls govern routing and service priorities. Finite horizon and long run average cost problems are considered and value iteration is a key tool. Nonconvex value functions are shown to exist for slightly more general networks. Nonconvex value functions are also shown to arise for a simple single station control problem in which the instantaneous cost is convex but not monotone. Nevertheless, optimality of threshold policies is established for the single station problem. The proof is based on a novel use of stochastic coupling and policy iteration.     

Adaptive decentralized control under non-uniqueness of the optimal control

We study the problem of decentralization of flow control in packet-switching networks under the isarithmic scheme. An incoming packet enters the network only if there are permits available at the entry port when it arrives. The actions of the controllers refer to the routing of permits in the network and the control variables are the corresponding probabilities. We study the behavior of adaptive algorithms implemented at the controllers to update these probabilities and seek optimal performance. This problem can be stated as a routing problem in a closed queueing network. The centralized version of a learning automation is a general framework presented along with the proof of asymptotic optimality. Decentralization of the controller gives rise to non-uniqueness of the optimal control parameters. Non-uniqueness can be exploited to construct asymptotically optimal learning algorithms that exhibit different behavior. We implement two different algorithms for the parallel operation and discuss their differences. Convergence is established using the weak convergence methodology. In addition to our theoretical results, we illustrate the main results using the flow control problem as a model example and verify the predicted behavior of the two proposed algorithms through computer simulations, including an example of tracking.The work of this author was partially supported by a grant from the Canadian Institute for Telecommunications Research under the NCE program of the Government of Canada, and partially supported by NSERC grant WFA 0139015 and FCAR-Québec grant 95-NC-1375.The work of this author was supported by a grant from the CITR under the NCE program of the Government of Canada.     

The Optimal Policy of Dividends, Investments, and Capital Distribution for the Dynamic Model of a Company

Optimization of a dynamic production-finance model of a company utilizing two technological processes for producing a homogeneous product is investigated. The problem is formulated as an optimal control problem for a nonlinear object with phase and mixed constraints. Optimal programs and feedback controls are derived from sufficient conditions of optimality.     

Adaptive back-pressure congestion control based on localinformation

The problem of distributed congestion control as it arises in communication networks as well as in manufacturing systems is studied in this paper. In particular, a multistage queueing system that models virtual circuit and datagram communication networks and a class of manufacturing systems are considered. The topology may be arbitrary, there are multiple traffic classes, and the routing can be class dependent, with routes that may form direct or indirect loops. The model incorporates the functions of transmission scheduling, flow control, and routing, through which congestion control is performed in the network. A policy is given that performs these functions jointly. According to the policy, heavily loaded queues are given higher priority in service. A congested node may reduce the how from upstream nodes through a flow control mechanism. Whenever routing is required, it is performed in such a manner that the lightly loaded queues receive most of the traffic. For arrival processes with bounded burstiness, the policy guarantees bounded backlogs in the network, as long as the load of each server is less than one. The actions of each server are based on the state of its own queues and of the queues one hop away. Therefore, they are implementable in a distributed fashion. An adaptive version of the policy is also provided which makes it independent of the arrival rates     

What is the optimal architecture for visual information routing?

Analyzing the design of networks for visual information routing is an underconstrained problem due to insufficient anatomical and physiological data. We propose here optimality criteria for the design of routing networks. For a very general architecture, we derive the number of routing layers and the fanout that minimize the required neural circuitry. The optimal fanout l is independent of network size, while the number k of layers scales logarithmically (with a prefactor below 1), with the number n of visual resolution units to be routed independently. The results are found to agree with data of the primate visual system.     

Global optimal feedback control for general nonlinear systems with nonquadratic performance criteria

Optimal control of general nonlinear nonaffine controlled systems with nonquadratic performance criteria (that permit state- and control-dependent time-varying weighting parameters), is solved classically using a sequence of linear- quadratic and time-varying problems. The proposed method introduces an “approximating sequence of Riccati equations” (ASRE) to explicitly construct nonlinear time-varying optimal state-feedback controllers for such nonlinear systems. Under very mild conditions of local Lipschitz continuity, the sequences converge (globally) to nonlinear optimal stabilizing feedback controls. The computational simplicity and effectiveness of the ASRE algorithm is an appealing alternative to the tedious and laborious task of solving the Hamilton–Jacobi–Bellman partial differential equation. So the optimality of the ASRE control is studied by considering the original nonlinear-nonquadratic optimization problem and the corresponding necessary conditions for optimality, derived from Pontryagin's maximum principle. Global optimal stabilizing state-feedback control laws are then constructed. This is compared with the optimality of the ASRE control by considering a nonlinear fighter aircraft control system, which is nonaffine in the control. Numerical simulations are used to illustrate the application of the ASRE methodology, which demonstrate its superior performance and optimality.     

Energetic sustainability of routing algorithms for energy-harvesting wireless sensor networks

A new class of wireless sensor networks that harvest power from the environment is emerging because of its intrinsic capability of providing unbounded lifetime. While a lot of research has been focused on energy-aware routing schemes tailored to battery-operated networks, the problem of optimal routing for energy harvesting wireless sensor networks (EH-WSNs) has never been explored. The objective of routing optimization in this context is not extending network lifetime, but maximizing the workload that can be autonomously sustained by the network.In this work we present a methodology for assessing the energy efficiency of routing algorithms for networks whose nodes drain power from the environment. We first introduce the energetic sustainability problem, then we define the maximum energetically sustainable workload (MESW) as the objective function to be used to drive the optimization of routing algorithms for EH-WSNs.We propose a methodology that makes use of graph algorithms and network simulations for evaluating the MESW starting from a network topology, a routing algorithm and a distribution of the environmental power available at each node. We present a tool flow implementing the proposed methodology and we show comparative results achieved on several routing algorithms.Experimental results highlight that routing strategies that do not take into account environmental power do not provide optimal results in terms of workload sustainability. Using optimal routing algorithms may lead to sizeable enhancements of the maximum sustainable workload. Moreover, optimality strongly depends on environmental power configurations. Since environmental power sources change over time, our results prompt for a new class of routing algorithms for EH-WSNs that are able to dynamically adapt to time-varying environmental conditions.     

Global optimality of approximate dynamic programming and its use in non-convex function minimization

This study investigates the global optimality of approximate dynamic programming (ADP) based solutions using neural networks for optimal control problems with fixed final time. Issues including whether or not the cost function terms and the system dynamics need to be convex functions with respect to their respective inputs are discussed and sufficient conditions for global optimality of the result are derived. Next, a new idea is presented to use ADP with neural networks for optimization of non-convex smooth functions. It is shown that any initial guess leads to direct movement toward the proximity of the global optimum of the function. This behavior is in contrast with gradient based optimization methods in which the movement is guided by the shape of the local level curves. Illustrative examples are provided with single and multi-variable functions that demonstrate the potential of the proposed method.     

Optimality and convergence of adaptive optimal control by reinforcement synthesis

Adaptive Optimal Control (AOC) by reinforcement synthesis is proposed to facilitate the application of optimal control theory in feedback controls. Reinforcement synthesis uses the critic–actor architecture of reinforcement learning to carry out sequential optimization. Optimality conditions for AOC are formulated using the discrete minimum principle. A proof of the convergence conditions for the reinforcement synthesis algorithm is presented. As the final time extends to infinity, the reinforcement synthesis algorithm is equivalent to the Dual Heuristic dynamic Programming (DHP) algorithm, a version of approximate dynamic programming. Thus, formulating DHP with the AOC approach has rigorous proofs of optimality and convergence. The efficacy of AOC by reinforcement synthesis is demonstrated by solving a linear quadratic regulator problem.     

On the convergence of nonlinear optimal control using pseudospectral methods for feedback linearizable systems

We consider the optimal control of feedback linearizable dynamic systems subject to mixed state and control constraints. In contrast to the existing results, the optimal controller addressed in this paper is allowed to be discontinuous. This generalization requires a substantial modification to the existing convergence analysis in terms of both the framework as well as the notion of convergence around points of discontinuity. Although the nonlinear system is assumed to be feedback linearizable, the optimal control does not necessarily linearize the dynamics. Such problems frequently arise in astronautical applications where stringent performance requirements demand optimality over feedback linearizing controls. We prove that a sequence of solutions obtained using the Legendre pseudospectral method converges to the optimal solution of the continuous‐time problem under mild conditions. Published in 2007 by John Wiley & Sons, Ltd.