Stochastic maximum principle in the mean-field controls

In Buckdahn, Djehiche, Li, and Peng (2009), the authors obtained mean-field Backward Stochastic Differential Equations (BSDEs) in a natural way as a limit of some highly dimensional system of forward and backward SDEs, corresponding to a great number of “particles” (or “agents”). The objective of the present paper is to deepen the investigation of such mean-field BSDEs by studying their stochastic maximum principle. This paper studies the stochastic maximum principle (SMP) for mean-field controls, which is different from the classical ones. This paper deduces an SMP in integral form, and it also gets, under additional assumptions, necessary conditions as well as sufficient conditions for the optimality of a control. As an application, this paper studies a linear quadratic stochastic control problem of mean-field type.     

Stochastic Motion and the Level Set Method in Computer Vision: Stochastic Active Contours

Based on recent work on Stochastic Partial Differential Equations (SPDEs), this paper presents a simple and well-founded method to implement the stochastic evolution of a curve. First, we explain why great care should be taken when considering such an evolution in a Level Set framework. To guarantee the well-posedness of the evolution and to make it independent of the implicit representation of the initial curve, a Stratonovich differential has to be introduced. To implement this differential, a standard Ito plus drift approximation is proposed to turn an implicit scheme into an explicit one. Subsequently, we consider shape optimization techniques, which are a common framework to address various applications in Computer Vision, like segmentation, tracking, stereo vision etc. The objective of our approach is to improve these methods through the introduction of stochastic motion principles. The extension we propose can deal with local minima and with complex cases where the gradient of the objective function with respect to the shape is impossible to derive exactly. Finally, as an application, we focus on image segmentation methods, leading to what we call Stochastic Active Contours.     

Identification and Control of Multivariable Stochastic Feedback Systems

Abstract

This paper considers the problem of simultaneous identification and control of stochastic processes characterized by linear dynamic models with unknown systems parameter coefficients. Stochastic approximation is used to derive consistent identification algorithms for the case in which arbitrary feedback controls are present. These identification methods can also be used for determining the order of the system, if the latter is unknown, as well as the exact canonical structure for the multivariable case.

An approximation to the optimal control solution is obtained by explicitly separating the functions of identification and control, and asymptotic convergence to a stochastic optimal controller is attained without on-line structural modification.     

Backward Stochastic H2/H∞ Control with Random Jumps

This paper is concerned with H₂/H_∞ control of a new class of stochastic systems. The most distinguishing feature, compared with the existing literature, is that the systems are described by backward stochastic differential equations (BSDEs) with Brownian motion and random jumps. It is shown that the backward stochastic H₂/H_∞ control under consideration is associated with the of the corresponding uncontrolled backward stochastic perturbed system. A necessary and sufficient condition for the existence of a unique solution to the control problem under consideration is derived. The resulting solution is characterized by the solution of an uncontrolled forward backward stochastic differential equation (FBSDE) with Brownian motion and random jumps. When the coefficients are all deterministic, the equivalent linear feedback solution involves a pair of Riccati‐type equations and an uncontrolled BSDE. In addition an uncontrolled forward stochastic differential equation (SDE) is given.     

Step decision rules for multistage stochastic programming: A heuristic approach

Stochastic programming with step decision rules (SPSDR) aims to produce efficient solutions to multistage stochastic optimization problems. SPSDR, like plain multistage Stochastic Programming (SP), operates on a Monte Carlo “computing sample” of moderate size that approximates the stochastic process. Unlike SP, SPSDR does not strive to build a balanced event tree out of that sample. Rather, it defines a solution as a special type of decision rule, with the property that the decisions at each stage are piecewise constant functions on the sample of scenarios. Those pieces define a partition of the set of scenarios at each stage t, but the partition at t+1 need not be refinement of the partition at t. However, the rule is constructed so that the non-anticipativity condition is met, a necessary condition to make the rules operational. To validate the method we show how to extend a non-anticipatory decision rule to arbitrary scenarios within a very large validation sample of scenarios. We apply three methods, SPSDR, SP and Robust Optimization, to the same 12-stage problem in supply chain management, and compare them relatively to different objectives and performance criteria. It appears that SPSDR performs better than SP in that it produces a more accurate estimate (prediction) of the value achieved by its solution on the validation sample, and also that the achieved value is better.     

Option pricing under a jump-telegraph diffusion model with jumps of random size

ABSTRACT

We present a finite difference method to solve a system of two Partial-Integro Differential Equations which arise from pricing an option under a Jump-Telegraph Diffusion Model for the underlying asset, considering the risk-neutral valuation formula under an equivalent martingale measure. This system is fully discretized using an Implicit–Explicit two-time level scheme and quadrature formulas. The resulting two tridiagonal algebraic linear systems are solved recursively using the Thomas Algorithm. Some numerical results are presented and the numerical order of convergence for the method is estimated. Finally, the robustness of the method is validated against an exact solution obtained for a perturbed problem.     

Hybrid dynamic iterations for the solution of initial value problems

Many scientific problems are posed as Ordinary Differential Equations (ODEs). A large subset of these are initial value problems, which are typically solved numerically. The solution starts by using a known state space of the ODE system to determine the state at a subsequent point in time. This process is repeated several times. When the computational demand is high due to large state space, parallel computers can be used efficiently to reduce the time to solution. Conventional parallelization strategies distribute the state space of the problem amongst cores and distribute the task of computing for a single time step amongst the cores. They are not effective when the computational problems have fine granularity, for example, when the state space is relatively small and the computational effort arises largely from the long time span of the initial value problem. We propose a hybrid dynamic iterations method¹ which combines conventional sequential ODE solvers with dynamic iterations to parallelize the time domain. Empirical results demonstrate a factor of two to four improvement in performance of the hybrid dynamic iterations method over a conventional ODE solver on an 8 core processor. Compared to Picard iterations (also parallelized in the time domain), the proposed method shows better convergence and speedup results when high accuracy is required.     

BENCHOP – SLV: the BENCHmarking project in Option Pricing – Stochastic and Local Volatility problems

ABSTRACT

In the recent project BENCHOP – the BENCHmarking project in Option Pricing we found that Stochastic and Local Volatility problems were particularly challenging. Here we continue the effort by introducing a set of benchmark problems for this type of problems. Eight different methods targeted for the Stochastic Differential Equation (SDE) formulation and the Partial Differential Equation (PDE) formulation of the problem, as well as Fourier methods making use of the characteristic function, were implemented to solve these problems. Comparisons are made with respect to time to reach a certain error level in the computed solution for the different methods. The implemented Fourier method was superior to all others for the two problems where it was implemented. Generally, methods targeting the PDE formulation of the problem outperformed the methods for the SDE formulation. Among the methods for the PDE formulation the ADI method stood out as the best performing one.     

Stochastic grid bundling method for backward stochastic differential equations

ABSTRACT

In this work, we apply the Stochastic Grid Bundling Method (SGBM) to numerically solve backward stochastic differential equations (BSDEs). The SGBM algorithm is based on conditional expectations approximation by means of bundling of Monte Carlo sample paths and a local regress-later regression within each bundle. The basic algorithm for solving the backward stochastic differential equations will be introduced and an upper error bound is established for the local regression. A full error analysis is also conducted for the explicit version of our algorithm and numerical experiments are performed to demonstrate various properties of our algorithm.     

Properties of stochastic syntax-directed translation schemata

Stochastic syntax-directed translation schemata describe both the syntactic structure and the probability distribution of stochastic mappings between contextfree languages. The relationship between stochastic syntax-directed translation schemata and stochastic grammars and automata are presented by proving that a stochastic pushdown transducer can be constructed to define the same translations as a simple schema, and that the simple schema are characterized by stochastic contextfree grammars. Asymptotic properties of linear schemata are established by the theory of Markov chains. Since stochastic translations contain both input and output strings, their information content can be described. Equations are developed for both the information content and the rate of stochastic translations.     

The zero divisor problem of multivariable stochastic adaptive control

Stochastic adaptive minimum variance control algorithms require a division by a function of a recursively computed parameter estimate at each instant of time. In order that the analysis of these algorithms is valid, zero divisions must be events of probability zero. This property is established for the stochastic gradient adaptive control algorithm under the condition that the initial state of the system and all finite segments of its random disturbance process have a joint distribution which is absolutely continuous with respect to Lebesgue measure. This result is deduced from the following general result established in this paper: a non-constant rational function of a finite set of random variables {x₁},x_n} is absolutely continuous with respect to Lebesgue measure if the joint distribution function of {x₁,…,x_n} has this property.     

Modeling And Performance Evaluation Of Branch And Value Prediction In Ilp Processors

Speculative execution is one of the key issues to boost the performance of future generation microprocessors. In this paper, we introduce a novel approach to evaluate the effects of branch and value prediction, which allow the processor to execute instructions beyond the limits of control and true data dependences. Until now, almost all the estimations of their performance potential under different scenarios have been obtained using trace-driven or execution-driven simulation. Occasionally, some simple deterministic models have been used. We employ an analytical model based on recently introduced Fluid Stochastic Petri Nets (FSPNs) in order to capture the dynamic behavior of an ILP processor with aggressive use of prediction techniques and speculative execution. Here we define the FSPN model, derive the state equations for the underlying stochastic process and present performance evaluation results to illustrate its usage in deriving measures of interest. Our implementation-independent stochastic modeling framework reveals considerable potential for further research in this area using numerical solution of systems of partial differential equations and/or discrete-event simulation of FSPN models.     

On Self-Learning Systems

Abstract

The fundamental problem on self-learning is discussed. The characteristics and the capability of two representative organizers obtained by the Karhunen-Loève system and the stochastic approximation method are described and their relationship with Bayes' solution is discussed.

The organization obtainable by the KL system is realized by calculating the difference of the features between the categories and by classifying the system so as to maximize this difference. This procedure is shown to be optimal. Further, the sufficient condition for this procedure to be the Bayes' solution is derived.

For the organization attainable by the stochastic approximation method, two functional to be evaluated are introduced and the system is organized so as to minimize these functionals. The necessary and the sufficient conditions for these procedures to be Bayes' solution are obtained for the mixture of normal distributions with identical covariance matrices.

The several simulations on a digital computer are performed for both cases. The results of these simulations indicate the validity of our learning systems.     

New criteria on exponential stability of neutral stochastic differential delay equations

Neutral stochastic differential delay equations (NSDDEs) have recently been studied intensively (see e.g. [V.B. Kolmanovskii, V.R. Nosov, Stability and Periodic Modes of Control Systems with Aftereffect, Nauka, Moscow, 1981; X. Mao, Exponential stability in mean square of neutral stochastic differential functional equations, Systems Control Lett. 26 (1995) 245–251; X. Mao, Razumikhin type theorems on exponential stability of neutral stochastic functional differential equations, SIAM J. Math. Anal. 28(2) (1997) 389–401; X. Mao, Stochastic Differential Equations and Their Applications, Horwood Publishing, Chichester, 1997]). More recently, Mao [Asymptotic properties of neutral stochastic differential delay equations, Stochastics and Stochastics Rep. 68 (2000) 273–295] provided with some useful criteria on the exponential stability for NSDDEs. However, the criteria there require not only the coefficients of the NSDDEs to obey the linear growth condition but also the time delay to be a constant. One of our aims in this paper is to remove these two restrictive conditions. Moreover, the key condition on the diffusion operator associated with the underlying NSDDE will take a much more general form. Our new stability criteria not only cover many highly non-linear NSDDEs with variable time delays but they can also be verified much more easily than the known criteria.     

libMoM : a library for stochastic simulations in engineering using statistical moments

Stochastic simulations are becoming increasingly important in numerous engineering applications. The solution to the governing equations are complicated due to the high-dimensional spaces and the presence of randomness. In this paper we present libMoM (), a software library to solve various types of Stochastic Differential Equations (SDE) as well as estimate statistical distributions from the moments. The library provides a suite of tools to solve various SDEs using the method of moments (MoM) as well as estimate statistical distributions from the moments using moment matching algorithms. For a large class of problems, MoM provide efficient solutions compared with other stochastic simulation techniques such as Monte Carlo (MC). In the physical sciences, the moments of the distribution are usually the primary quantities of interest. The library enables the solution of moment equations derived from a variety of SDEs, with closure using non-standard Gaussian quadrature. In engineering risk assessment and decision making, statistical distributions are required. The library implements tools for fitting the Generalized Lambda Distribution (GLD) with the given moments. The objectives of this paper are (1) to briefly outline the theory behind moment methods for solving SDEs/estimation of statistical distributions; (2) describe the organization of the software and user interfaces; (3) discuss use of standard software engineering tools for regression testing, aid collaboration, distribution and further development. A number of representative examples of the use of libMoM in various engineering applications are presented and future areas of research are discussed.     

Static and output-feedback of discrete-time LTI systems with state multiplicative noise

A parameter dependent approach for designing static output-feedback controller for linear time-invariant systems with state-multiplicative noise is introduced which achieves a minimum bound on either the stochastic H₂ or the H_∞ performance levels. A solution is obtained also for the case where, in addition to the stochastic parameters, the system matrices reside in a given polytope. In this case, a parameter dependent Lyapunov function is described which enables the derivation of the required constant feedback gain via a solution of a set of linear matrix inequalities that correspond to the vertices of the uncertainty polytope.The stochastic parameters appear in both the dynamics and the input matrices of the state space model of the system. The problems are solved using the expected value of the standard performance indices over the stochastic parameters. The theory developed is demonstrated by a simple example.     

Performance Evaluation of a Real-time Simulation Architecture using Probabilistic Model Checking

The goal of this paper is to show how to use probabilistic model checking techniques in order to achieve quantitative performance evaluation of a real-time distributed simulation. A simulation based on the High Level Architecture (HLA) is modelled as a stochastic process, a Continuous Time Markov Chain (CTMC), using the stochastic algebra PEPA. Next a property representing a performance constraint is evaluated applying Continuous Stochastic Logic CSL formula on the CTMC model using the probabilistic model checker PRISM. Finally a first experiment is made to compare the model with a real case.