Enhancements of two-stage stochastic decomposition

This paper presents some enhancements associated with stochastic decomposition (SD). Specifically, we study two issues: (a) Are there any conditions under which the regularized version of SD generates a unique solution? (b) Is there a way to modify the SD algorithm so that a user can trade-off solution times with solution quality? The second issue addresses the scalability of SD for very large scale problems for which computational resources may be limited and the user may be willing to accept solutions that are “nearly optimal”. We show that by using bootstrapping (re-sampling) the regularized SD algorithm can be accelerated without significant loss of optimality. We report computational results that demonstrate the viability of this approach.     

Confidence level solutions for stochastic programming

We propose an alternative approach to stochastic programming based on Monte-Carlo sampling and stochastic gradient optimization. The procedure is by essence probabilistic and the computed solution is a random variable. We propose a solution concept in which the probability that the random algorithm produces a solution with an expected objective value departing from the optimal one by more than ? is small enough. We derive complexity bounds on the number of iterations of this process. We show that by repeating the basic process on independent samples, one can significantly reduce the number of iterations.     

The zero divisor problem of multivariable stochastic adaptive control

Stochastic adaptive minimum variance control algorithms require a division by a function of a recursively computed parameter estimate at each instant of time. In order that the analysis of these algorithms is valid, zero divisions must be events of probability zero. This property is established for the stochastic gradient adaptive control algorithm under the condition that the initial state of the system and all finite segments of its random disturbance process have a joint distribution which is absolutely continuous with respect to Lebesgue measure. This result is deduced from the following general result established in this paper: a non-constant rational function of a finite set of random variables {x₁},x_n} is absolutely continuous with respect to Lebesgue measure if the joint distribution function of {x₁,…,x_n} has this property.     

Model checking mobile stochastic logic

The Temporal Mobile Stochastic Logic (MoSL) has been introduced in previous work by the authors for formulating properties of systems specified in StoKlaim, a Markovian extension of Klaim. The main purpose of MoSL is to address key functional aspects of global computing such as distribution awareness, mobility, and security and their integration with performance and dependability guarantees. In this paper, we present MoSL⁺${\text{<small-caps>MoSL</small-caps>}}^{+}$, an extension of MoSL, which incorporates some basic features of the Modal Logic for MObility (MoMo), a logic specifically designed for dealing with resource management and mobility aspects of concurrent behaviours. We also show how MoSL⁺${\text{<small-caps>MoSL</small-caps>}}^{+}$ formulae can be model-checked against StoKlaim specifications. For this purpose, we show how existing state-based stochastic model-checkers, like e.g. the Markov Reward Model Checker (MRMC), can be exploited by using a front-end for StoKlaim that performs appropriate pre-processing of MoSL⁺${\text{<small-caps>MoSL</small-caps>}}^{+}$ formulae. The proposed approach is illustrated by modelling and verifying a sample system.     

Detectability and observability of discrete-time stochastic systems and their applications

This paper studies detectability and observability of discrete-time stochastic linear systems. Based on the standard notions of detectability and observability for time-varying linear systems, corresponding definitions for discrete-time stochastic systems are proposed which unify some recently reported detectability and exact observability concepts for stochastic linear systems. The notion of observability leads to the stochastic version of the well-known rank criterion for observability of deterministic linear systems. By using these two concepts, the discrete-time stochastic Lyapunov equation and Riccati equations are studied. The results not only extend some of the existing results on these two types of equation but also indicate that the notions of detectability and observability studied in this paper take analogous functions as the usual concepts of detectability and observability in deterministic linear systems. It is expected that the results presented may play important roles in many design problems in stochastic linear systems.     

A two-stage approach to the orienteering problem with stochastic weights

The Orienteering Problem (OP) is a routing problem which has many interesting applications in logistics, tourism and defense. The aim of the OP is to find a maximum profit path or tour, which is feasible with respect to a capacity constraint on the total weight of the selected arcs. In this paper we consider the Orienteering Problem with Stochastic Weights (OPSWs) to reflect uncertainty in real-life applications. We approach this problem by formulating a two-stage stochastic model with recourse for the OPSW where the capacity constraint is hard. The model takes into account the effect that stochastic weights have on the expected total profit value to be obtained, already in the modeling stage. Since the expected profit is in general non-linear, we introduce a linearization that models the total profit that can be obtained for a given tour and a given scenario of weight realizations. This linearization allows for the application of Sample Average Approximation (SAA). The SAA solution asymptotically converges to the optimal solution of the two-stage model, but is computationally expensive. Therefore, to solve large instances, we developed a heuristic that exploits the problem structure of the OPSW and explicitly takes the associated uncertainty into account. In our computational experiments, we evaluate the benefits of our approach to the OPSW, compared to both a standard deterministic approach, and a deterministic approach that is extended with utilization of real-time information.     

Solutions to a class of linear-quadratic-Gaussian (LQG) stochastic team problems with nonclassical information

We consider a stochastic dynamic team problem with two controllers and nonclassical information, which can be viewed as the transmission of a garbled version of a Gaussian message over a number of noisy channels under a fidelity criterion. We show that the optimum solution (under a quadratic loss functional) consists of linearly transforming the garbled message to a certain (optimum) power level P^* and then optimally decoding it by using a linear transformation at the receiving end. The power level P^* is determined by the solution of a fifth order algebraic equation. The paper also discusses an extension of this result to the case when the channel noise is correlated with the input random variable, and shows that for the single channel case the optimum solution is again linear.     

Detectability in stochastic discrete event systems

A discrete event system possesses the property of detectability if it allows an observer to perfectly estimate the current state of the system after a finite number of observed symbols, i.e., detectability captures the ability of an observer to eventually perfectly estimate the system state. In this paper we analyze detectability in stochastic discrete event systems (SDES) that can be modeled as probabilistic finite automata. More specifically, we define the notion of A-detectability, which characterizes our ability to estimate the current state of a given SDES with increasing certainty as we observe more output symbols. The notion of A-detectability is differentiated from previous notions for detectability in SDES because it takes into account the probability of problematic observation sequences (that do not allow us to perfectly deduce the system state), whereas previous notions for detectability in SDES considered each observation sequence that can be generated by the underlying system. We discuss observer-based techniques that can be used to verify A-detectability, and provide associated necessary and sufficient conditions. We also prove that A-detectability is a PSPACE-hard problem.     

The maximum principle for stochastic differential systems with general cost functional

In this paper, under the framework of Fréchet derivatives, we study a stochastic optimal control problem driven by a stochastic differential equation with general cost functional. By constructing a series of first-order and second-order adjoint equations, we establish the stochastic maximum principle and get the related Hamilton systems.     

Confidence tori in the analysis of stochastic 3D-cycles

We present a new computer approach to the spatial analysis of stochastically forced 3D-cycles in nonlinear dynamic systems. This approach is based on a stochastic sensitivity analysis and uses the construction of confidence tori. A confidence torus as a simple 3D-model of the stochastic cycle adequately describes its main probabilistic features. We suggest an effective algorithm for construction of the confidence tori using a discrete set of confidence ellipses. The ability of these tori to visualize thin effects observed for the period-doubling bifurcations zone in the stochastic Roessler model are shown. For this zone, the geometrical growth of stochastic sensitivity of the forced cycles under transition to chaos is presented.