Interactive visualization of large state spaces

Insight into the global structure of a state space is of great help in the analysis of the underlying process. We advocate the use of visualization for this purpose and present a method to visualize the structure of very large state spaces with millions of nodes. The method uses a clustering based on an equivalence relation to obtain a simplified representation, which is used as a backbone for the display of the entire state space. With this visualization we are able to answer questions about the global structure of a state space that cannot easily be answered by conventional methods. We show this by presenting a number of visualizations of real-world protocols .     

Exploring very large state spaces using genetic algorithms

We present a novel framework for exploring very large state spaces of concurrent reactive systems. Our framework exploits application-independent heuristics using genetic algorithms to guide a state-space search toward error states. We have implemented this framework in conjunction with VeriSoft, a tool for exploring the state spaces of software applications composed of several concurrent processes executing arbitrary code. We present experimental results obtained with several examples of programs, including a C implementation of a public-key authentication protocol. We discuss heuristics and properties of state spaces that help a genetic search detect deadlocks and assertion violations. For finding errors in very large state spaces, our experiments show that a genetic search using simple heuristics can significantly outperform random and systematic searches.     

Search strategy for scheduling flexible manufacturing systems simultaneously using admissible heuristic functions and nonadmissible heuristic functions

To scheduling flexible manufacturing system (FMS) efficiently, we propose and evaluate an improved search strategy and its application to FMS scheduling in the P-timed Petri net framework. On the execution of Petri net, the proposed method can simultaneously use admissible heuristic functions and nonadmissible heuristic functions for A^* algorithm. We also prove that the resulting combinational heuristic function is still admissible and more informed than any of its constituents. The experimental results of an example FMS and several sets of random generated problems show that the proposed search method performs better as we expected.     

A probabilistic dynamic technique for the distributed generation of very large state spaces

Conventional methods for state space exploration are limited to the analysis of small systems because they suffer from excessive memory and computational requirements. We have developed a new dynamic probabilistic state exploration algorithm which addresses this problem for general, structurally unrestricted state spaces.

Our method has a low state omission probability and low memory usage that is independent of the length of the state vector. In addition, the algorithm can be easily parallelised. This combination of probability and parallelism enables us to rapidly explore state spaces that are an order of magnitude larger than those obtainable using conventional exhaustive techniques.

We derive a performance model of this new algorithm in order to quantify its benefits in terms of distributed run-time, speedup and efficiency. We implement our technique on a distributed-memory parallel computer and demonstrate results which compare favourably with the performance model. Finally, we discuss suitable choices for the three hash functions upon which our algorithm is based.     

Learning via human feedback in continuous state and action spaces

This paper considers the problem of extending Training an Agent Manually via Evaluative Reinforcement (TAMER) in continuous state and action spaces. Investigative research using the TAMER framework enables a non-technical human to train an agent through a natural form of human feedback (negative or positive). The advantages of TAMER have been shown on tasks of training agents by only human feedback or combining human feedback with environment rewards. However, these methods are originally designed for discrete state-action, or continuous state-discrete action problems. This paper proposes an extension of TAMER to allow both continuous states and actions, called ACTAMER. The new framework utilizes any general function approximation of a human trainer’s feedback signal. Moreover, a combined capability of ACTAMER and reinforcement learning is also investigated and evaluated. The combination of human feedback and reinforcement learning is studied in both settings: sequential and simultaneous. Our experimental results demonstrate the proposed method successfully allowing a human to train an agent in two continuous state-action domains: Mountain Car and Cart-pole (balancing).     

Potential functions based sampling heuristic for optimal path planning

Rapidly-exploring Random Tree star (RRT*) is a recently proposed extension of Rapidly-exploring Random Tree (RRT) algorithm that provides a collision-free, asymptotically optimal path regardless of obstacles geometry in a given environment. However, one of the limitation in the RRT* algorithm is slow convergence to optimal path solution. As a result it consumes high memory as well as time due to the large number of iterations utilised in achieving optimal path solution. To overcome these limitations, we propose the potential function based-RRT* that incorporates the artificial potential field algorithm in RRT*. The proposed algorithm allows a considerable decrease in the number of iterations and thus leads to more efficient memory utilization and an accelerated convergence rate. In order to illustrate the usefulness of the proposed algorithm in terms of space execution and convergence rate, this paper presents rigorous simulation based comparisons between the proposed techniques and RRT* under different environmental conditions. Moreover, both algorithms are also tested and compared under non-holonomic differential constraints.     

Sequential stratified regeneration: MCMC for large state spaces with an application to subgraph count estimation

Data Mining and Knowledge Discovery - This work considers the general task of estimating the sum of a bounded function over the edges of a graph, given neighborhood query access and where access to...     

HSM2: A new heuristic state minimization algorithm for finite state machine

This paper proposes a heuristic state minimization algorithm (HSM2) for finite state machines (FSM). HSM2 focuses on the generation and adjustment of the closed cover. First an initial closed cover is generated by heuristically selecting proper maximal compatibles to satisfy all the covering and closure conditions, and then it is adjusted to be a minimal or near minimal closed cover by heuristically removing repeated states. Experimental results show that the algorithm is faster and obtains better or the same solutions compared with conventional methods.     

An adaptive large neighborhood search heuristic for the share-a-ride problem

The Share-a-Ride Problem (SARP) aims at maximizing the profit of serving a set of passengers and parcels using a set of homogeneous vehicles. We propose an adaptive large neighborhood search (ALNS) heuristic to address the SARP. Furthermore, we study the problem of determining the time slack in a SARP schedule. Our proposed solution approach is tested on three sets of realistic instances. The performance of our heuristic is benchmarked against a mixed integer programming (MIP) solver and the Dial-a-Ride Problem (DARP) test instances. Compared to the MIP solver, our heuristic is superior in both the solution times and the quality of the obtained solutions if the CPU time is limited. We also report new best results for two out of twenty benchmark DARP instances.     

A large neighborhood search heuristic for supply chain network design

Many exact and approximate solution techniques have been used to solve facility location problems and, more generally, supply chain network design problems. Yet, the Large Neighborhood Search technique (LNS) has almost never been suggested for solving such problems, although it has proven its efficiency and flexibility in solving other complex combinatorial optimization problems. In this paper, we propose an LNS framework for solving a four-layer single period multi-product supply chain network design problem. One important feature of the model is that it includes inter-modality: the itinerary followed by the cargo from origin to destination may take several transportation modes. Moreover, several modes may compete on some arcs. Location decisions for intermediate facilities (e.g. plants and distribution centers) are determined by the LNS while transportation modes and product flow decisions are determined by a greedy heuristic. As a post-optimization step, linear programming is used to optimize product flows once the structure of the logistics network is fixed. Extensive experiments, based on randomly generated instances of different sizes and characteristics, show the effectiveness of the method compared with a state-of-the-art solver.     

An aggregation heuristic for large scale p-median problem

The p-median problem (PMP) consists of locating p facilities (medians) in order to minimize the sum of distances from each client to the nearest facility. The interest in the large-scale PMP arises from applications in cluster analysis, where a set of patterns has to be partitioned into subsets (clusters) on the base of similarity.In this paper we introduce a new heuristic for large-scale PMP instances, based on Lagrangean relaxation. It consists of three main components: subgradient column generation, combining subgradient optimization with column generation; a “core” heuristic, which computes an upper bound by solving a reduced problem defined by a subset of the original variables chosen on a base of Lagrangean reduced costs; and an aggregation procedure that defines reduced size instances by aggregating together clients with the facilities. Computational results show that the proposed heuristic is able to compute good quality lower and upper bounds for instances up to 90,000 clients and potential facilities.     

Condensed state spaces for symmetrical Coloured Petri Nets

This paper deals with state spaces. A state space is a directed graph with a node for each reachable state and an arc for each possible state change. We describe how symmetries of the modelled system can be exploited to obtain much more succinct state space analysis. The symmetries induce equivalence classes of states and equivalence classes of state changes. It is then possible to construct a condensed state space where each node represents an equivalence class of states while each arc represents an equivalence class of state changes. Such a condensed state space is often much smaller than the full state space and it is also much faster to construct. Nevertheless, it is possible to use the condensed state space to verify the same kind of behavioural properties as the full state space. hence, we do not lose analytic power.We define state spaces and condensed state spaces for a language called Coloured Petri Nets (CP-nets). This language is in widespread use for the modelling and analysis of concurrent systems. However, our techniques are general and they can be used for many other kinds of labelled transition systems. The paper does not assume that the reader is familiar with CP-nets (or Petri nets in general)—although such knowledge will, of course, be a help. The first four sections of the paper introduce the basic concepts of CP-nets. The next three sections deal with state spaces, condensed state spaces and computer tools for state space analysis. Finally, there is a short conclusion.     

Parametric POMDPs for planning in continuous state spaces

This work addresses the problem of decision-making under uncertainty for robot navigation. Since robot navigation is most naturally represented in a continuous domain, the problem is cast as a continuous-state POMDP. Probability distributions over state space, or beliefs, are represented in parametric form using low-dimensional vectors of sufficient statistics. The belief space, over which the value function must be estimated, has dimensionality equal to the number of sufficient statistics. Compared to methods based on discretising the state space, this work trades the loss of the belief space’s convexity for a reduction in its dimensionality and an efficient closed-form solution for belief updates. Fitted value iteration is used to solve the POMDP. The approach is empirically compared to a discrete POMDP solution method on a simulated continuous navigation problem. We show that, for a suitable environment and parametric form, the proposed method is capable of scaling to large state-spaces.     

A general system for heuristic minimization of convex functions over non-convex sets

We describe general heuristics to approximately solve a wide variety of problems with convex objective and decision variables from a non-convex set. The heuristics, which employ convex relaxations, convex restrictions, local neighbour search methods, and the alternating direction method of multipliers, require the solution of a modest number of convex problems, and are meant to apply to general problems, without much tuning. We describe an implementation of these methods in a package called NCVX, as an extension of CVXPY, a Python package for formulating and solving convex optimization problems. We study several examples of well known non-convex problems, and show that our general purpose heuristics are effective in finding approximate solutions to a wide variety of problems.     

Simplex filter: A novel heuristic filter for nonlinear systems state estimation

This paper introduces a new filter for nonlinear systems state estimation. The new filter formulates the state estimation problem as a stochastic dynamic optimization problem and utilizes a new stochastic method based on simplex technique to find and track the best estimation. The vertices of the simplex search the state space dynamically in a similar scheme to the optimization algorithm, known as Nelder-Mead simplex. The parameters of the proposed filter are tuned, using an information visualization technique to identify the optimal region of the parameters space. The visualization is performed using the concept of parallel coordinates. The proposed filter is applied to estimate the state of some nonlinear dynamic systems with noisy measurement and its performance is compared with other filters.     

An adaptive large neighborhood search heuristic for the Electric Vehicle Scheduling Problem

This paper addresses the Electric Vehicle Scheduling Problem (E-VSP), in which a set of timetabled bus trips, each starting from and ending at specific locations and at specific times, should be carried out by a set of electric buses or vehicles based at a number of depots with limited driving ranges. The electric vehicles are allowed to be recharged fully or partially at any of the given recharging stations. The objective is to firstly minimize the number of vehicles needed to cover all the timetabled trips, and secondly to minimize the total traveling distance, which is equivalent to minimizing the total deadheading distance. A mixed integer programming formulation as well as an Adaptive Large Neighborhood Search (ALNS) heuristic for the E-VSP are presented. ALNS is tested on newly generated E-VSP benchmark instances. Result shows that the proposed heuristic can provide good solutions to large E-VSP instances and optimal or near-optimal solutions to small E-VSP instances.     

General sampling theorems for functions in reproducing kernel Hilbert spaces

In this paper we prove general sampling theorems for functions belonging to a reproducing kernel Hilbert space (RKHS) which is also a closed subspace of a particular Sobolev space. We present details of this approach as applied to the standard sampling theory and its extension to nonuniform sampling. The general theory for orthogonal sampling sequences and nonorthogonal sampling sequences is developed. Our approach includes as concrete cases many recent extensions, for example, those based on the Sturm-Liouville transforms, Jacobi transforms, Laguerre transforms, Hankel transforms, prolate spherical transforms, etc., finite-order sampling theorems, as well as new sampling theorems obtained by specific choices of the RKHS. In particular, our setting includes nonorthogonal sampling sequences based on the theory of frames. The setting and approach enable us to consider various types of errors (truncation, aliasing, jitter, and amplitude error) in the same general context. This work was supported in part by the National Science Foundation under Grant DMS-901526.     

Optimal compression of combinatorial state spaces

Innovations in Systems and Software Engineering - Efficiently deciding reachability for model checking problems requires storing the entire state space. We provide an information theoretical lower...     

An adaptive large neighborhood search heuristic for the cumulative capacitated vehicle routing problem

The cumulative capacitated vehicle routing problem (CCVRP) is a variation of the classical capacitated vehicle routing problem in which the objective is the minimization of the sum of arrival times at customers, instead of the total routing cost. This paper presents an adaptive large neighborhood search heuristic for the CCVRP. This algorithm is applied to a set of benchmark instances and compared with two recently published memetic algorithms.     

An adaptive large neighborhood search heuristic for solving a robust gate assignment problem

With the rapid growth of air traffic demand, airport capacity becomes a major bottleneck within the air traffic control systems. Minor disturbances may have a large impact on the airport surface operations due to the overly tight schedules, which results in frequent gate conflict occurrences during airport’s daily operations. A robust gate schedule that is resilient to disturbances is essential for an airport to maintain a good performance. Unfortunately, there is no efficient expert system available for the airport managers to simultaneously consider the traditional cost (the aircraft tow cost, transfer passenger cost) and the robustness. To fill this gap, in this paper, we extend the traditional gate assignment problem and consider a wider scope, in which the traditional costs and the robustness are simultaneously considered. A mathematical model is first built, which leads to a complex non-linear model. To efficiently solve this model, an adaptive large neighborhood search (ALNS) algorithm is then designed. We novelly propose multiple local search operators by exploring the characteristics of the gate assignment problem. The comparison with the benchmark algorithm shows the competitiveness of proposed algorithm in solving the considered problem. Moreover, the proposed methodology also has great potential from the practical perspective since it can be easily integrated into current expert systems to help airport managers make satisfactory decisions.