Constraint-based probabilistic modeling for statistical abduction

We introduce a new framework for logic-based probabilistic modeling called constraint-based probabilistic modeling which defines CBPMs (constraint-based probabilistic models) , i.e. conditional joint distributions P(⋅∣KB) over independent propositional variables constrained by a knowledge base KB consisting of clauses. We first prove that generative models such as PCFGs and discriminative models such as CRFs have equivalent CBPMs as long as they are discrete. We then prove that CBPMs in infinite domains exist which give existentially closed logical consequences of KB probability one. Finally we derive an EM algorithm for the parameter learning of CBPMs and apply it to statistical abduction.     

On convex parameterization of robust control design for minimizing (conditional) performance at risk

This paper introduces performance at risk and conditional performance at risk as design metrics for the formulation of robust control design. These two metrics are used to characterize the high percentile or tail distribution of a performance specification when system uncertain parameters are random variables described by statistical distributions. The probabilistic robust control design is then formulated as a minimization problem with respect to the (conditional) performance at risk or as a constrained problem in terms of them. Performance specifications in terms of the high percentile or tail distribution are more stringent than that are defined in terms of the average (mean) value, which are often used in current literature for probabilistic robust control. Furthermore, the convexity of the conditional performance at risk does not have particular requirements on the underlying distribution of uncertain parameters; thus, convex optimization can be applied to the probabilistic robust control with respect to uncertain parameters with general distributions. The proposed probabilistic robust approach is applied to search solutions to linear matrix inequality containing random parametric uncertainties as well as to design a stabilizing controller for polynomial vector fields subject to random parametric uncertainties. Copyright © 2007 John Wiley & Sons, Ltd.     

H∞ output feedback control for stochastic systems with randomly occurring convex‐bounded uncertainties and channel fadings

In this paper, the H_∞ output feedback control problem for a class of stochastic discrete‐time systems with randomly occurring convex‐bounded uncertainties and channel fadings is investigated. A sequence of mutually independent random variables with known probabilistic distributions are utilized to describe the randomness that convex‐bounded uncertainties appear in practical systems. The measurements with channel fadings are given by a stochastic Rice fading model which is regulated by a set of random variables with certain probability density functions. The purpose of this paper is to design an output feedback controller such that the closed‐loop control system is asymptotically stable with a prescribed H_∞ performance level. The less conservative results are obtained by employing the stochastic Lyapunov technique. Numerical examples are presented to illustrate effectiveness of the proposed approach.     

EXPLOITING SYNTACTIC,SEMANTIC, AND LEXICAL REGULARITIES IN LANGUAGE MODELING VIA DIRECTED MARKOV RANDOM FIELDS

We present a directed Markov random field (MRF) model that combines n‐gram models, probabilistic context‐free grammars (PCFGs), and probabilistic latent semantic analysis (PLSA) for the purpose of statistical language modeling. Even though the composite directed MRF model potentially has an exponential number of loops and becomes a context‐sensitive grammar, we are nevertheless able to estimate its parameters in cubic time using an efficient modified Expectation‐Maximization (EM) method, the generalized inside–outside algorithm, which extends the inside–outside algorithm to incorporate the effects of the n‐gram and PLSA language models. We generalize various smoothing techniques to alleviate the sparseness of n‐gram counts in cases where there are hidden variables. We also derive an analogous algorithm to find the most likely parse of a sentence and to calculate the probability of initial subsequence of a sentence, all generated by the composite language model. Our experimental results on the Wall Street Journal corpus show that we obtain significant reductions in perplexity compared to the state‐of‐the‐art baseline trigram model with Good–Turing and Kneser–Ney smoothing techniques.     

‘Perfectly’ Scalable Data I/O

This paper describes and analyzes a new architecture for file systems in which ‘metadata’, lock control, etc., are distributed among diverse resources. The basic data structure is a segment, viz. a logical group of files, folders, or other objects. The file system requires only one root, and can be non-hierarchical without a complete tree structure within segments. For ‘embarrassingly parallel’ data distributions, scalability is trivially perfect for all N,where N is the number of servers. Even for random file access, a new extreme statistical mechanics is used to show that data I/O is ‘perfectly’ scalable with probability 1, with degradation from perfect scaling that is small and bounded by f ln N/ ln (ln N). Here f is the fraction of data that is metadata. In contrast, earlier solutions degrade much faster, like Nf. No structural changes in classical metadata are required.     

Model checking for a probabilistic branching time logic with fairness

We consider concurrent probabilistic systems, based on probabilistic automata of Segala & Lynch [55], which allow non-deterministic choice between probability distributions. These systems can be decomposed into a collection of “computation trees” which arise by resolving the non-deterministic, but not probabilistic, choices. The presence of non-determinism means that certain liveness properties cannot be established unless fairness is assumed. We introduce a probabilistic branching time logic PBTL, based on the logic TPCTL of Hansson [30] and the logic PCTL of [55], resp. pCTL [14]. The formulas of the logic express properties such as “every request is eventually granted with probability at least p”. We give three interpretations for PBTL on concurrent probabilistic processes: the first is standard, while in the remaining two interpretations the branching time quantifiers are taken to range over a certain kind of fair computation trees. We then present a model checking algorithm for verifying whether a concurrent probabilistic process satisfies a PBTL formula assuming fairness constraints. We also propose adaptations of existing model checking algorithms for pCTL [4, 14] to obtain procedures for PBTL under fairness constraints. The techniques developed in this paper have applications in automatic verification of randomized distributed systems. Received: June 1997 / Accepted: May 1998     

Probabilistic robust design with linear quadratic regulators

In this paper, we study robust design of uncertain systems in a probabilistic setting by means of linear quadratic regulators (LQR). We consider systems affected by random bounded nonlinear uncertainty so that classical optimization methods based on linear matrix inequalities cannot be used without conservatism. The approach followed here is a blend of randomization techniques for the uncertainty together with convex optimization for the controller parameters. In particular, we propose an iterative algorithm for designing a controller which is based upon subgradient iterations. At each step of the sequence, we first generate a random sample and then we perform a subgradient step for a convex constraint defined by the LQR problem. The main result of the paper is to prove that this iterative algorithm provides a controller which quadratically stabilizes the uncertain system with probability one in a finite number of steps. In addition, at a fixed step, we compute a lower bound of the probability that a quadratically stabilizing controller is found.     

Testing Preorders for Probabilistic Processes

We present a testing preorder for probabilistic processes based on a quantification of the probability with which processes pass tests. The theory enjoys close connections with the classical testing theory of De Nicola and Hennessy in that whenever a process passes a test with probability 1 (respectively some nonzero probability) in our setting, then the process must (respectively may) pass the test in the classical theory. We also develop an alternative characterization of the probabilistic testing preorders that takes the form of a mapping from probabilistic traces to the interval [0, 1], where a probabilistic trace is an alternating sequence of actions and probability distributions over actions. Finally, we give proof techniques, derived from the alternative characterizations, for establishing preorder relationships between probabilistic processes. The utility of these techniques is demonstrated by means of some simple examples.     

Formalization of the Standard Uniform random variable

Continuous random variables are widely used to mathematically describe random phenomena in engineering and the physical sciences. In this paper, we present a higher-order logic formalization of the Standard Uniform random variable as the limit value of the sequence of its discrete approximations. We then show the correctness of this specification by proving the corresponding probability distribution properties within the HOL theorem prover, summarizing the proof steps. The formalized Standard Uniform random variable can be transformed to formalize other continuous random variables, such as Uniform, Exponential, Normal, etc., by using various non-uniform random number generation techniques. The formalization of these continuous random variables will enable us to perform an error free probabilistic analysis of systems within the framework of a higher-order-logic (HOL) theorem prover. For illustration purposes, we present the formalization of the Continuous Uniform random variable based on the formalized Standard Uniform random variable, and then utilize it to perform a simple probabilistic analysis of roundoff error in HOL.     

Recursive identification for Wiener systems using Gaussian inputs

The recursive algorithms are given for identifying the single‐input single‐output Wiener system which consists of a moving average type linear subsystem followed by a static nonparametric nonlinearity. The input is defined to be a sequence of mutually independent Gaussian random variables. The estimates for coefficients of the linear subsystem as well as for f(v) at any v are proved to converge to the true values with probability one. A numerical example is given, justifying the theoretical analysis. Copyright © 2008 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Simulating perfect channels with probabilistic lossy channels

We consider the problem of deciding whether an infinite-state system (expressed as a Markov chain) satisfies a correctness property with probability 1. This problem is, of course, undecidable for general infinite-state systems. We focus our attention on the model of probabilistic lossy channel systems consisting of finite-state processes that communicate over unbounded lossy FIFO channels. Abdulla and Jonsson have shown that safety properties are decidable while progress properties are undecidable for non-probabilistic lossy channel systems. Under assumptions of “sufficiently high” probability of loss, Baier and Engelen have shown how to check whether a property holds of probabilistic lossy channel system with probability 1. In this paper, we consider a model of probabilistic lossy channel systems, where messages can be lost only during send transitions. In contrast to the model of Baier and Engelen, once a message is successfully sent to channel, it can only be removed through a transition which receives the message. We show that checking whether safety properties hold with probability 1 is undecidable for this model. Our proof depends upon simulating a perfect channel, with a high degree of confidence, using lossy channels.     

Probability‐Generated Aggregators

The paper addresses a relation between logical reasoning and probability and presents probability‐generated aggregators. The obtained aggregators implement probability distributions for specification of generator functions; as it was proven in the paper, such implementation is always possible. In the paper, the relation between neutral element of the probabilistic uninorm and parameters of the underlying probability distribution is demonstrated, and a method for specification of the probabilistic uninorm, and thus—of the probability distribution using t‐norm and t‐conorm—is constructed. In addition, the obtained probabilistic uninorm and probabilistic absorbing norm or nullnorm are briefly considered as algebraic operations on the open unit interval. In is demonstrated, that, in general, the obtained algebra is nondistributive and depends on the distributions, which are used for generating probabilistic uninorm and absorbing norm. The obtained results bridge several gaps between fuzzy and probabilistic logics and provide a basis both for theoretical studies in the field and for practical techniques of digital/analog schemes synthesis and analysis.     

On Probabilistic Networks for Selection, Merging, and Sorting

We study comparator networks for selection, merging, and sorting that output the correct result with high probability, given a random input permutation. We prove tight bounds, up to constant factors, on the size and depth of probabilistic (n,k)-selection networks. In the case of (n, n/2)-selection, our result gives a somewhat surprising bound of on the size of networks of success probability in , where δ is an arbitrarily small positive constant, thus comparing favorably with the best previously known solutions, which have size . We also prove tight bounds, up to lower-order terms, on the size and depth of probabilistic merging networks of success probability in , where δ is an arbitrarily small positive constant. Finally, we describe two fairly simple probabilistic sorting networks of success probability at least and nearly logarithmic depth. Received January 22, 1996, and in final form February 14, 1997.     

Probabilistic single obnoxious facility location with fixed budget

This paper investigates probabilistic single obnoxious facility location with fixed budget which is defined as locating the facility to maximize the probability that the minimum weighted distance from the facility to all non-expropriated demand nodes exceeds a given threshold and the maximum weighted distance from the facility to all expropriated demand nodes does not exceed another given value, where demand weights are random variables with general continuous probability distributions. Properties of the optimal solutions are identified and heuristic solution procedures are presented, especially under the condition of some specific probability distributions. The general problem we propose also leads to some known problems such as maximin, quantile location problems.     

Informational independence: Models and normal forms

The concept of informational independence plays a key role in most knowledge-based systems. J. Pearl and his co-researchers analysed the basic properties of the concept and formulated an axiomatic system for informational independence. This axiomatic system focuses on independences among mutually disjoint sets of variables. We show that in the context of probabilistic independence a focus on disjoint sets of variables can hide various interesting properties. To capture these properties, we enhance Pearl's axiomatic system with two additional axioms. We investigate the set of models of the thus enhanced system and show that it provides a better characterization of the concept of probabilistic independence than Pearl's system does. In addition, we observe that both Pearl's axiomatic system and our enhanced system offer inference rules for deriving new independences from an initial set of independence statements and as such allow for a normal form for representing independence. We address the normal forms ensuing from the two axiomatic systems for informational independence. © 1998 John Wiley & Sons, Inc.13: 83–109, 1998     

Optimal quota allocation for a revenue-maximizing auction holder facing a random number of bidders

This paper is motivated by the following practical multi‐unit auction. Before purchasing a vehicle in Singapore, a prospective buyer must bid for a certificate that entitles him to purchase the vehicle. Each month a certain number, say Q, of “certificates of entitlement” are made available and the highest Q bidders receive the right to purchase. Each successful bidder pays a “quota premium” equal to the bid submitted by the Qth highest bidder. We construct a stochastic optimization model using results from the theory of order statistics, where the objective is to determine the optimal number of units to be released in order to maximize the auction holders expected revenue. As the amounts bid and the number of bidders are both random, there may be wide – and, undesirable – fluctuations in the quota premia from one month to the next. We perform the optimization subject to a probabilistic constraint that limits the month‐to‐month fluctuations in quota premia. We show, assuming specific distributions for the bid amounts and the number of bidders, that in order to satisfy the probabilistic constraint one may have to reduce the available quota. For different bid random variables, the feasible set arising from the probabilistic constraint may assume different shapes. We also present a sufficient condition for comparing the size of the two feasible sets corresponding to different bid random variables. The results are illustrated by sensitivity analyses that examine the effect of varying parameter values on the optimal solution.     

Regular Random k-SAT: Properties of Balanced Formulas

We consider a model for generating random k-SAT formulas, in which each literal occurs approximately the same number of times in the formula clauses (regular random and k-SAT). Our experimental results show that such regular random k-SAT instances are much harder than the usual uniform random k-SAT problems. This is in agreement with other results that show that more balanced instances of random combinatorial problems are often much more difficult to solve than uniformly random instances, even at phase transition boundaries. There are almost no formal results known for such problem distributions. The balancing constraints add a dependency between variables that complicates a standard analysis. Regular random 3-SAT exhibits a phase transition as a function of the ratio α of clauses to variables. The transition takes place at approximately α = 3.5. We show that for α > 3.78 with high probability (w.h.p.) random regular 3-SAT formulas are unsatisfiable. Specifically, the events hold with high probability if Pr when n → ∞. We also show that the analysis of a greedy algorithm proposed by Kaporis et al. for the uniform 3-SAT model can be adapted for regular random 3-SAT. In particular, we show that for formulas with ratio α < 2.46, a greedy algorithm finds a satisfying assignment with positive probability.     

Probabilistic assessment of small disturbance stability in multimachine power systems

In planning electric power systems, it is always necessary to assess whether small-disturbance (SD) instability phenomena occur at prefixed system operating conditions. This analysis can become very difficult when the problem data are uncertain. In such cases the use of deterministic approaches is inadequate and the application of probabilistic analysis techniques is the most feasible alternative. This paper presents a new and practical probabilistic approach for the assessment of SD stability in multimachine power systems taking into account the uncertainties associated with bus load forecasting and treating loads as random uncorrelated variables with normal distributions. This approach proves suitable for determining the risk of SD instability for each expected system operating condition and for systematically individualizing all factors that can affect the probability of SD instability in large power systems. A numerical example illustrates the capability of the proposed technique.     

The application of Kolmogorov–Rényi statistics to problems of parameter uncertainty in systems design†

This paper presents a method for dealing with parameter uncertainty in system design which is based on the study of the statistical properties of an ensemble of systems defined by a given structure and by a priori parameter distributions rather than point parameter estimates. It is assumed that the model of the actual system is a random member of the ensemble. The object of the analysis is to design or modify the properties of the ensemble to ensure a high probability of adequate performance of the actual system. The primary statistical function employed is the sample distribution function. This function is used to estimate the true population distribution of a scalar variable chosen to measure the system property of interest. The sample distribution function is constructed from random samples of this figure of merit generated by a suitable digital computer programme. The accuracy of the estimation of the population distribution by the sample distribution is determined by application of statistical results of Kolmogorov and Rényi.