Decentralized MDPs with sparse interactions

Creating coordinated multiagent policies in environments with uncertainty is a challenging problem, which can be greatly simplified if the coordination needs are known to be limited to specific parts of the state space. In this work, we explore how such local interactions can simplify coordination in multiagent systems. We focus on problems in which the interaction between the agents is sparse and contribute a new decision-theoretic model for decentralized sparse-interaction multiagent systems, Dec-SIMDPs, that explicitly distinguishes the situations in which the agents in the team must coordinate from those in which they can act independently. We relate our new model to other existing models such as MMDPs and Dec-MDPs. We then propose a solution method that takes advantage of the particular structure of Dec-SIMDPs and provide theoretical error bounds on the quality of the obtained solution. Finally, we show a reinforcement learning algorithm in which independent agents learn both individual policies and when and how to coordinate. We illustrate the application of the algorithms throughout the paper in several multiagent navigation scenarios.     

Decentralized multi-robot allocation of tasks with temporal and precedence constraints

Abstract

We present an auction-based method for a team of robots to allocate and execute tasks that have temporal and precedence constraints. Temporal constraints are expressed as time windows, within which a task must be executed. The robots use our priority-based iterated sequential single-item auction algorithm to allocate tasks among themselves and keep track of their individual schedules. A key innovation is in decoupling precedence constraints from temporal constraints and dealing with them separately. We demonstrate the performance of the allocation method and show how it can be extended to handle failures and delays during task execution. We leverage the power of simulation as a tool to analyze the robustness of schedules. Data collected during simulations are used to compute well-known indexes that measure the risk of delay and failure in the robots’ schedules. We demonstrate the effectiveness of our method in simulation and with real robot experiments.     

Solving project scheduling problems with resource constraints via an event list-based evolutionary algorithm

There are various scheduling problems with resource limitations and constraints in the literature that can be modeled as variations of the Resource Constrained Project Scheduling Problem (RCPSP). This paper proposes a new solution representation and an evolutionary algorithm for solving the RCPSP. The representation scheme is based on an ordered list of events, that are sets of activities that start (or finish) at the same time. The proposed solution methodology, namely SAILS, operates on the event list and relies on a scatter search framework. The latter incorporates an Adaptive Iterated Local Search (AILS), as an improvement method, and integrates an event-list based solution combination method. AILS utilizes new enriched neighborhoods, guides the search via a long term memory and applies an efficient perturbation strategy. Computational results on benchmark instances of the literature indicate that both AILS and SAILS produce consistently high quality solutions, while the best results are derived for most problem data sets.     

Decentralized control of heavy-haul trains with input constraints and communication delays

The heavy-haul trains consist of a large number of locomotives and wagons which usually cover a very long range of train tracks with different slopes and curvatures. As a consequence, the control design for the heavy-haul trains has posed a significant challenge due to the modeling complexity and the complicated traveling conditions, under which the problems of communication delays and input constraints naturally arise. In this paper, a novel decentralized control design is proposed for the heavy-haul trains by explicitly addressing the issues of input constraints and time-varying communication delays. Specifically, the complicated longitudinal dynamical model of the heavy-haul trains is first converted into a double integrator form by introducing a set of state and input transformations. Then, a decentralized cooperative control is proposed to regulate the speeds of individual train units (locomotives and wagons) to the desired profile with the objective of maintaining the minimum in-train force for safety consideration. It is rigorously proved that the closed-loop system is uniformly asymptotically cooperative stable under the least communication conditions among individual train units. Extensive simulations are conducted to validate the performance of the proposed new design.     

Validation of protocols with temporal constraints

Reachability analysis is the most popular and most used method in protocol validation. It consists in constructing a graph called a reachability graph, describing communication of machines exchanging messages through FIFO channels. The states and structure of this graph are then analysed according to given properties to validate the related communication protocol. In this paper, we deal with a generalization of reachability analysis to take into account quantitative temporal constraints in protocol validation. A temporal reachability graph is designed, and we show how it can be used to analyse new general properties of communication protocols submitted to temporal constraints.     

Decentralized bandwidth control subject to two-layer constraints

This paper is concerned with the new bandwidth allocation model that considers the structural allocation constraints. Suppose that a system is composed of finite groups of users and the bandwidth assigned to each user and to each group of users has pre-specified upper and lower bounds. If each user is granted with the utility function satisfying the standard continuity and concavity conditions, the existence, uniqueness, fairness and optimality properties of the Nash equilibrium point in the allocation game are studied. To identify the equilibrium point, an algorithm proved to converge globally is proposed and illustrated with a numerical example.     

Uncertain project scheduling problem with resource constraints

Project scheduling problem is to make a schedule for allocating the loans to a project such that the total cost and the completion time of the project are balanced under some constraints. This paper presents an uncertain project scheduling problem, of which both the duration times and the resources allocation times are uncertain variables. An uncertain programming model with multiple objectives is obtained, whose first objective is to minimize the total cost, and second objective is to minimize the overtime. Genetic algorithm is employed to solve the proposed uncertain project scheduling model, and its efficiency is illustrated by a numerical experiment.     

Formalizing requirements with object models and temporal constraints

Flaws in requirements often have a negative impact on the subsequent development phases. In this paper, we present a novel approach for the formal representation and validation of requirements, which we used in an industrial project. The formalism allows us to represent and reason about object models and their temporal evolution. The key ingredients are class diagrams to represent classes of objects, their relationships and their attributes, fragments of first order logic to constrain the possible configurations of such objects, and temporal logic operators to deal with the dynamic evolution of the configurations. The approach to formal validation allows to check whether the requirements are consistent, if they are compatible with some scenarios, and if they guarantee some implicit properties. The validation procedure is based on satisfiability checking, which is carried out by means of finite instantiation and model checking techniques.     

Decentralized task allocation for heterogeneous multi-UAV system with task coupling constraints

The Journal of Supercomputing - Cooperative multiple task assignment problem is an essential issue in the collaboration of multiple unmanned aerial vehicles (UAVs). Consensus-based bundle algorithm...     

Minimizing mean flow-time with parallel processors and resource constraints

Summary The problem to be considered is one of scheduling nonpreemptable tasks in multiprocessor systems when tasks need for their processing processors and other limited resources, and when mean flow time is the system performance measure. For each task the time required for its processing and the amount of each resource which it requires, are given. Special attention is paid to the computational complexity of algorithms for determining the optimal schedules for different assumptions concerning the environment. For the case of scheduling independent, arbitrary length tasks when each task may require a unit of an additional resource of one type, an O(n ³) algorithm is given. For more complicated resource requirements, however, it is proved that the problem under consideration is NP-hard in the strong sense, even for the case of two processors.     

Integrated QoS-aware resource management and scheduling with multi-resource constraints

In dynamic real-time systems such as sensor networks, mobile ad hoc networking and autonomous systems, the mapping between level of service and resource requirements is often not fixed. Instead, the mapping depends on a combination of level of service and outside environmental factors over which the application has no direct control. An example of an application where environmental factors play a significant role is radar tracking. In radar systems, resources must be shared by a set of radar tasks including tracking, searching and target confirmation tasks. Environmental factors such as noise, heating constraints of the radar and the speed, distance and maneuverability of tracked targets dynamically affect the mapping between the level of service and resource requirements. The QoS manager in a radar system must be adaptive, responding to dynamic changes in the environment by efficiently reallocating resource to maintain an acceptable level of service. In this paper, we present an integrated QoS optimization and dwell scheduling scheme for a radar tracking application. QoS optimization is performed using the Q-RAM (Baugh, 1973, ghosh-et al.,2004a approach. Heuristics are used to achieve a two order magnitude of reduction in optimization time over the basic Q-RAM approach allowing QoS optimization and scheduling of a 100 task radar problem to be performed in as little as 700 ms with only a 0.1% QoS penality over Q-RAM alone. Sourav Ghosh received the B.Tech degree in Electronics and Electrical Communications Engineering from Indian Institute of Technology, Kharagpur, India, in 1997, and the M.S. and Ph.D. degrees in Electrical and Computer Engineering from Carnegie Mellon University, Pittsburgh, PA, USA, in 1999 and 2004 respectively. He is currently working as a Technical Staff at Oracle in Cluster Database Group (RAC). His research interest includes OS resource management and scheduling, performance analysis, Quality of Service (QoS) and real-time systems.     

Solving two-machine assembly scheduling problems with inventory constraints

This paper considers a scheduling problem with component availability constraints in a supply chain consisting of two manufacturing facilities and a merge-in-transit facility. Three mixed-integer programming (MIP) models and a constraint programming (CP) model are compared in an extensive numerical study. Results show that when there are no components shared among the two manufacturers, the MIP model based on time-index variables is the best for proving optimality for problems with short processing times whereas the CP model tends to perform better than the others for problems with a large range of processing times. When shared components are added, the performance of all models deteriorates, with the time-indexed MIP providing the best results. By explicitly modelling the dependence of scheduling decisions on the availability of component parts, we contribute to the literature on the integration of inventory and scheduling decisions, which is necessary for solving realistic supply chain problems.     

Solving coalitional resource games

Coalitional Resource Games (crgs) are a form of Non-Transferable Utility (ntu) game, which provide a natural formal framework for modelling scenarios in which agents must pool scarce resources in order to achieve mutually satisfying sets of goals. Although a number of computational questions surrounding crgs have been studied, there has to date been no attempt to develop solution concepts for crgs, or techniques for constructing solutions. In this paper, we rectify this omission. Following a review of the crg framework and a discussion of related work, we formalise notions of coalition structures and the core for crgs, and investigate the complexity of questions such as determining nonemptiness of the core. We show that, while such questions are in general computationally hard, it is possible to check the stability of a coalition structure in time exponential in the number of goals in the system, but polynomial in the number of agents and resources. As a consequence, checking stability is feasible for systems with small or bounded numbers of goals. We then consider constructive approaches to generating coalition structures. We present a negotiation protocol for crgs, give an associated negotiation strategy, and prove that this strategy forms a subgame perfect equilibrium. We then show that coalition structures produced by the protocol satisfy several desirable properties: Pareto optimality, dummy player, and pseudo-symmetry.     

Solving hierarchies of finite-domain constraints

Abstract. In the past we presented an algorithm to solve ordered constraint hierarchies based on a non-trivial error function using standard constraint satisfaction techniques. We extended this previous work and herewith present a new method to transform hierarchies of inequalities over the integers based on global comparators into equivalent ordered constraint hierarchies. The correctness of this method is proven. Using the results of our previous work, we present another method transforming the resulting ordered constraint hierarchies into ordinary constraint systems. These systems of algebraic equalities and inequalities over the integer domain are soluble with available finite-domain constraint solvers. Finally, we propose some modifications and simplifications of the considered constraint hierarchies, improving the search for solutions and being useful in practical applications like job-shop scheduling. The modifications proposed are the combination of consecutive hierarchy levels in one level where the constraints are considered with different priorities weights and the simplifications proposed are incomplete search strategies resulting in run-time improvements and also in sub-optimal solutions.     

Solving geometric constraints by homotopy

Numerous methods have been proposed in order to solve geometric constraints, all of them having their own advantages and drawbacks. In this article, we propose an enhancement to the classical numerical methods, which, up to now, are the only ones that apply to the general case     

Dynamic scheduling of design activities with resource constraints

A design process can be represented as a network of design activities. A number of design projects may be undertaken simultaneously. This paper deals with the problem of scheduling design activities of multiple design projects competing for the limited available resources. The problem of determining a schedule subject to precedence and resource constraints is difficult to solve. It becomes even more complex when unforeseen changes are considered, for example, in the level of resources. Therefore, the scheduling problem is decomposed into a series of multidimensional (multiresource) knapsack problems. Due to high computational complexity of the multidimensional knapsack problem, two solution procedures are proposed     

First-order temporal pattern mining with regular expression constraints

Previous studies on mining sequential patterns have focused on temporal patterns specified by some form of propositional temporal logic. However, there are some interesting sequential patterns, such as the multi-sequential patterns, whose specification needs a more expressive formalism, the first-order temporal logic. Multi-sequential patterns appear in different application contexts, for instance in spatial census data mining, which is the target application of the study developed in this paper. We extend a well-known user-controlled tool, based on regular expressions constraints, to the multi-sequential pattern context. This specification tool enables the incorporation of user focus into the mining process. We present MSP-Miner, an Apriori-based algorithm to discover all frequent multi-sequential patterns satisfying a user-specified regular expression constraint.