Decentralized MDPs with sparse interactions

Creating coordinated multiagent policies in environments with uncertainty is a challenging problem, which can be greatly simplified if the coordination needs are known to be limited to specific parts of the state space. In this work, we explore how such local interactions can simplify coordination in multiagent systems. We focus on problems in which the interaction between the agents is sparse and contribute a new decision-theoretic model for decentralized sparse-interaction multiagent systems, Dec-SIMDPs, that explicitly distinguishes the situations in which the agents in the team must coordinate from those in which they can act independently. We relate our new model to other existing models such as MMDPs and Dec-MDPs. We then propose a solution method that takes advantage of the particular structure of Dec-SIMDPs and provide theoretical error bounds on the quality of the obtained solution. Finally, we show a reinforcement learning algorithm in which independent agents learn both individual policies and when and how to coordinate. We illustrate the application of the algorithms throughout the paper in several multiagent navigation scenarios.     

Decentralized multi-robot allocation of tasks with temporal and precedence constraints

Abstract

We present an auction-based method for a team of robots to allocate and execute tasks that have temporal and precedence constraints. Temporal constraints are expressed as time windows, within which a task must be executed. The robots use our priority-based iterated sequential single-item auction algorithm to allocate tasks among themselves and keep track of their individual schedules. A key innovation is in decoupling precedence constraints from temporal constraints and dealing with them separately. We demonstrate the performance of the allocation method and show how it can be extended to handle failures and delays during task execution. We leverage the power of simulation as a tool to analyze the robustness of schedules. Data collected during simulations are used to compute well-known indexes that measure the risk of delay and failure in the robots’ schedules. We demonstrate the effectiveness of our method in simulation and with real robot experiments.     

Solving project scheduling problems with resource constraints via an event list-based evolutionary algorithm

There are various scheduling problems with resource limitations and constraints in the literature that can be modeled as variations of the Resource Constrained Project Scheduling Problem (RCPSP). This paper proposes a new solution representation and an evolutionary algorithm for solving the RCPSP. The representation scheme is based on an ordered list of events, that are sets of activities that start (or finish) at the same time. The proposed solution methodology, namely SAILS, operates on the event list and relies on a scatter search framework. The latter incorporates an Adaptive Iterated Local Search (AILS), as an improvement method, and integrates an event-list based solution combination method. AILS utilizes new enriched neighborhoods, guides the search via a long term memory and applies an efficient perturbation strategy. Computational results on benchmark instances of the literature indicate that both AILS and SAILS produce consistently high quality solutions, while the best results are derived for most problem data sets.     

Decentralized control of heavy-haul trains with input constraints and communication delays

The heavy-haul trains consist of a large number of locomotives and wagons which usually cover a very long range of train tracks with different slopes and curvatures. As a consequence, the control design for the heavy-haul trains has posed a significant challenge due to the modeling complexity and the complicated traveling conditions, under which the problems of communication delays and input constraints naturally arise. In this paper, a novel decentralized control design is proposed for the heavy-haul trains by explicitly addressing the issues of input constraints and time-varying communication delays. Specifically, the complicated longitudinal dynamical model of the heavy-haul trains is first converted into a double integrator form by introducing a set of state and input transformations. Then, a decentralized cooperative control is proposed to regulate the speeds of individual train units (locomotives and wagons) to the desired profile with the objective of maintaining the minimum in-train force for safety consideration. It is rigorously proved that the closed-loop system is uniformly asymptotically cooperative stable under the least communication conditions among individual train units. Extensive simulations are conducted to validate the performance of the proposed new design.     

Validation of protocols with temporal constraints

Reachability analysis is the most popular and most used method in protocol validation. It consists in constructing a graph called a reachability graph, describing communication of machines exchanging messages through FIFO channels. The states and structure of this graph are then analysed according to given properties to validate the related communication protocol. In this paper, we deal with a generalization of reachability analysis to take into account quantitative temporal constraints in protocol validation. A temporal reachability graph is designed, and we show how it can be used to analyse new general properties of communication protocols submitted to temporal constraints.     

Formalizing requirements with object models and temporal constraints

Flaws in requirements often have a negative impact on the subsequent development phases. In this paper, we present a novel approach for the formal representation and validation of requirements, which we used in an industrial project. The formalism allows us to represent and reason about object models and their temporal evolution. The key ingredients are class diagrams to represent classes of objects, their relationships and their attributes, fragments of first order logic to constrain the possible configurations of such objects, and temporal logic operators to deal with the dynamic evolution of the configurations. The approach to formal validation allows to check whether the requirements are consistent, if they are compatible with some scenarios, and if they guarantee some implicit properties. The validation procedure is based on satisfiability checking, which is carried out by means of finite instantiation and model checking techniques.     

Uncertain project scheduling problem with resource constraints

Project scheduling problem is to make a schedule for allocating the loans to a project such that the total cost and the completion time of the project are balanced under some constraints. This paper presents an uncertain project scheduling problem, of which both the duration times and the resources allocation times are uncertain variables. An uncertain programming model with multiple objectives is obtained, whose first objective is to minimize the total cost, and second objective is to minimize the overtime. Genetic algorithm is employed to solve the proposed uncertain project scheduling model, and its efficiency is illustrated by a numerical experiment.     

一类资源约束的单机成组调度问题

本文讨论具有连续资源的单机成组调度问题.在这一模型中,工件组的安装时间是所消耗资源的非负严格减少连续函数,工件的加工时间是开工时间的严格增加函数.考虑两个问题,第1个问题是在满足资源消耗总量限制条件下,极小化最大完工时间.第2个问题的目标函数是在满足最大完工时间限制条件下,极小化资源消耗总量.分别对两个问题讨论了最优调度的某些特征,分别给出了求解最优资源分配的方法,并通过数值例子进行说明.     

Decentralized bandwidth control subject to two-layer constraints

This paper is concerned with the new bandwidth allocation model that considers the structural allocation constraints. Suppose that a system is composed of finite groups of users and the bandwidth assigned to each user and to each group of users has pre-specified upper and lower bounds. If each user is granted with the utility function satisfying the standard continuity and concavity conditions, the existence, uniqueness, fairness and optimality properties of the Nash equilibrium point in the allocation game are studied. To identify the equilibrium point, an algorithm proved to converge globally is proposed and illustrated with a numerical example.     

具有未建模动态和输出约束的耦合系统的分散自适应控制

针对一类具有输入、状态未建模动态和非线性输入的耦合系统,提出一种自适应神经网络控制方案.利用径向基函数神经网络逼近未知非线性连续函数;引入动态信号和正则化信号处理状态及输入未建模动态;通过引入非线性映射,将具有时变输出约束的严格反馈系统化为不含约束的严格反馈系统.最后,通过理论分析验证闭环系统中所有信号是半全局一致最终有界的,仿真结果进一步验证了所提出控制方案的有效性.     

Solving problems with unilateral constraints by DAE methods

We study trajectories of mechanical systems with unilateral constraints under the additional assumption that always a given number of constraints is active. A reformulation as a problem with bilateral conditions yields a drastic reduction in the number of constraints, but in general, we are faced with regularity problems. We illustrate our approach in the special case of a dynamical rigid body contact problem. In particular, we present a regularization technique which leads to the definition of generalized solutions and a quite effective numerical method on the basis of algorithms for differential–algebraic systems. The results are applied to a wheel–rail contact problem of actual interest to railway engineers.     

Decentralized task allocation for heterogeneous multi-UAV system with task coupling constraints

The Journal of Supercomputing - Cooperative multiple task assignment problem is an essential issue in the collaboration of multiple unmanned aerial vehicles (UAVs). Consensus-based bundle algorithm...     

Minimizing mean flow-time with parallel processors and resource constraints

Summary The problem to be considered is one of scheduling nonpreemptable tasks in multiprocessor systems when tasks need for their processing processors and other limited resources, and when mean flow time is the system performance measure. For each task the time required for its processing and the amount of each resource which it requires, are given. Special attention is paid to the computational complexity of algorithms for determining the optimal schedules for different assumptions concerning the environment. For the case of scheduling independent, arbitrary length tasks when each task may require a unit of an additional resource of one type, an O(n ³) algorithm is given. For more complicated resource requirements, however, it is proved that the problem under consideration is NP-hard in the strong sense, even for the case of two processors.     

Integrated QoS-aware resource management and scheduling with multi-resource constraints

In dynamic real-time systems such as sensor networks, mobile ad hoc networking and autonomous systems, the mapping between level of service and resource requirements is often not fixed. Instead, the mapping depends on a combination of level of service and outside environmental factors over which the application has no direct control. An example of an application where environmental factors play a significant role is radar tracking. In radar systems, resources must be shared by a set of radar tasks including tracking, searching and target confirmation tasks. Environmental factors such as noise, heating constraints of the radar and the speed, distance and maneuverability of tracked targets dynamically affect the mapping between the level of service and resource requirements. The QoS manager in a radar system must be adaptive, responding to dynamic changes in the environment by efficiently reallocating resource to maintain an acceptable level of service. In this paper, we present an integrated QoS optimization and dwell scheduling scheme for a radar tracking application. QoS optimization is performed using the Q-RAM (Baugh, 1973, ghosh-et al.,2004a approach. Heuristics are used to achieve a two order magnitude of reduction in optimization time over the basic Q-RAM approach allowing QoS optimization and scheduling of a 100 task radar problem to be performed in as little as 700 ms with only a 0.1% QoS penality over Q-RAM alone. Sourav Ghosh received the B.Tech degree in Electronics and Electrical Communications Engineering from Indian Institute of Technology, Kharagpur, India, in 1997, and the M.S. and Ph.D. degrees in Electrical and Computer Engineering from Carnegie Mellon University, Pittsburgh, PA, USA, in 1999 and 2004 respectively. He is currently working as a Technical Staff at Oracle in Cluster Database Group (RAC). His research interest includes OS resource management and scheduling, performance analysis, Quality of Service (QoS) and real-time systems.     

数据依赖约束下的任务调度资源选择算法

大数据环境下的计算任务往往具有一定数据依赖性关系(如MapReduce),现有的分布式存储系统任务资源选择策略选择离请求者最近的数据块响应服务,忽略了对数据块所在服务器CPU、磁盘I/O与网络等资源负载状态的考虑。在分析研究系统集群结构、文件分块、数据块存储机制的基础上,定义了集群节点矩阵、CPU负载矩阵、磁盘I/O负载矩阵、网络负载矩阵、文件分块矩阵、数据块存储矩阵与数据块存储节点状态矩阵,为任务与数据之间的依赖性构建了基础数据模型,提出了一种数据依赖约束下的最优资源选择算法(ORS2DC)。任务调度节点负责维护基础数据,MapReduce任务与数据块读取任务由于依赖资源不同而采取不同的选择策略。实验结果表明：所提算法能够为任务选择质量更高的资源,提高任务完成质量的同时减轻了NameNode负担,减小了单点故障发生的概率。     

Solving two-machine assembly scheduling problems with inventory constraints

This paper considers a scheduling problem with component availability constraints in a supply chain consisting of two manufacturing facilities and a merge-in-transit facility. Three mixed-integer programming (MIP) models and a constraint programming (CP) model are compared in an extensive numerical study. Results show that when there are no components shared among the two manufacturers, the MIP model based on time-index variables is the best for proving optimality for problems with short processing times whereas the CP model tends to perform better than the others for problems with a large range of processing times. When shared components are added, the performance of all models deteriorates, with the time-indexed MIP providing the best results. By explicitly modelling the dependence of scheduling decisions on the availability of component parts, we contribute to the literature on the integration of inventory and scheduling decisions, which is necessary for solving realistic supply chain problems.     

资源约束系统的控制与调度协同优化方法

对于一类通信约束的网络控制系统, 应用通信序列的概念和混合逻辑动态的构架, 将其建模成一类集成控制和调度的资源约束系统模型. 采用线性矩阵不等式来描述离散周期系统的周期通信序列和H_∞ 控制的协同优化问题, 并用一种递增优化序列算法来求解优化的调度策略及其相应的渐近稳定及r指数稳定的H_∞控制器. 该算法结合线性矩阵不等式凸优化问题解决了调度与控制的协同设计问题, 同时也节省了搜索优化解的计算时间.