Tests for Gaussian graphical models

Gaussian graphical models are promising tools for analysing genetic networks. In many applications, biologists have some knowledge of the genetic network and may want to assess the quality of their model using gene expression data. This is why one introduces a novel procedure for testing the neighborhoods of a Gaussian graphical model. It is based on the connection between the local Markov property and conditional regression of a Gaussian random variable. Adapting recent results on tests for high-dimensional Gaussian linear models, one proves that the testing procedure inherits appealing theoretical properties. Besides, it applies and is computationally feasible in a high-dimensional setting: the number of nodes may be much larger than the number of observations. A large part of the study is devoted to illustrating and discussing applications to simulated data and to biological data.     

Evolutionary optimization using graphical models

We have previously shown that a genetic algorithm can be approximated by an evolutionary algorithm using the product of univariate marginal distributions of selected points as search distribution. This algorithm (UMDA) successfully optimizes difficult multi-modal optimization problems. For correlated fitness landscapes more complex factorizations of the search distribution have to be used. These factorizations are used by the Factorized Distribution Algorithm FDA. In this paper we extend FDA to an algorithm which computes a factorization from the data. The factorization can be represented by a Bayesian network. The Bayesian network is used to generate the search points. Heinz Mühlenbein, Ph.D.: He is a research manager at GMD, the German national center for information technology. He obtained his diploma in mathematics from the University of Cologne in 1969, and his Ph.D from the University of Bonn in 1975. He entered GMD in 1969. He has worked on performance analysis of computer systems, computer networks, and massively parallel computers. Since 1988 his research interests are in Natural Computation. He was Visiting Professor at the Universities Paderborn, Bonn, Edinburgh and Carnegie-Mellon University. He has published over 60 research papers. He initiated the international conference series in natural computation PPSN in 1990. From 1993 to 1998 he was responsible European editor of Evolutionary Computation. He is presently on the Editorial Board of Evolutionary Computation, Scientific Computation and Journal of Heuristics. Thilo Mahnig, Ph.D. student: He is working at GMD — German National Research Center for Information Technology in St. Augustin. He obtained his diploma in mathematics from the University of Bonn in differential geometry in 1996. His research interest lies in the theory of population based optimization algorithms. He has co-authored several papers with Heinz Mühlenbein.     

OGM: Online gaussian graphical models on the fly

Applied Intelligence - Gaussian Graphical Model is widely used to understand the dependencies between variables from high-dimensional data and can enable a wide range of applications such as...     

A tighter bound for graphical models

We present a method to bound the partition function of a Boltzmann machine neural network with any odd-order polynomial. This is a direct extension of the mean-field bound, which is first order. We show that the third-order bound is strictly better than mean field. Additionally, we derive a third-order bound for the likelihood of sigmoid belief networks. Numerical experiments indicate that an error reduction of a factor of two is easily reached in the region where expansion-based approximations are useful.     

Towards graphical models for text processing

The continuous development of the Linked Data Web depends on the advancement of the underlying extraction mechanisms. This is of particular interest for the scientific publishing domain, where currently most of the data sets are being created manually. In this article, we present a Machine Learning pipeline that enables the automatic extraction of heading metadata (i.e., title, authors, etc) from scientific publications. The experimental evaluation shows that our solution handles very well any type of publication format and improves the average extraction performance of the state of the art with around 4%, in addition to showing an increased versatility. Finally, we propose a flexible Linked Data-driven mechanism to be used both for refining and linking the automatically extracted metadata.     

Learning possibilistic graphical models from data

Graphical models - especially probabilistic networks like Bayes networks and Markov networks - are very popular to make reasoning in high-dimensional domains feasible. Since constructing them manually can be tedious and time consuming, a large part of recent research has been devoted to learning them from data. However, if the dataset to learn from contains imprecise information in the form of sets of alternatives instead of precise values, this learning task can pose unpleasant problems. In this paper, we survey an approach to cope with these problems, which is not based on probability theory as the more common approaches like, e.g., expectation maximization, but uses the possibility theory as the underlying calculus of a graphical model. We provide semantic foundations of possibilistic graphical models, explain the rationale of possibilistic decomposition as well as the graphical representation of decompositions of possibility distributions and finally discuss the main approaches to learn possibilistic graphical models from data.     

Bayesian graphical models for software testing

This paper describes a new approach to the problem of software testing. The approach is based on Bayesian graphical models and presents formal mechanisms for the logical structuring of the software testing problem, the probabilistic and statistical treatment of the uncertainties to be addressed, the test design and analysis process, and the incorporation and implication of test results. Once constructed, the models produced are dynamic representations of the software testing problem. They may be used to drive test design, answer what-if questions, and provide decision support to managers and testers. The models capture the knowledge of the software tester for further use. Experiences of the approach in case studies are briefly discussed     

Uncertainty representation in software models: a survey

Software and Systems Modeling - This paper provides a comprehensive overview and analysis of research work on how uncertainty is currently represented in software models. The survey presents the...     

Regional category parsing in undirected graphical models

This article presents an approach for regional categorization in complex natural scenes with undirected graphs. A novel MRF-like model is proposed with spatial constraints in the feature space based on existing directed graphs, and an approximation of pseudo-likelihood is introduced for probability inference and parameter estimation. With this approximation, we can deal with the intractability of potential functions and get spatial relations between patches of different classes for more information in their co-occurrence matrix. The Receiver-Operating-Characteristic curves in our experiments demonstrate a better performance from our proposed method in comparison with directed probabilistic models such as LDA and constellation.     

Efficiently adapting graphical models for selectivity estimation

Query optimizers rely on statistical models that succinctly describe the underlying data. Models are used to derive cardinality estimates for intermediate relations, which in turn guide the optimizer to choose the best query execution plan. The quality of the resulting plan is highly dependent on the accuracy of the statistical model that represents the data. It is well known that small errors in the model estimates propagate exponentially through joins, and may result in the choice of a highly sub-optimal query execution plan. Most commercial query optimizers make the attribute value independence assumption: all attributes are assumed to be statistically independent. This reduces the statistical model of the data to a collection of one-dimensional synopses (typically in the form of histograms), and it permits the optimizer to estimate the selectivity of a predicate conjunction as the product of the selectivities of the constituent predicates. However, this independence assumption is more often than not wrong, and is considered to be the most common cause of sub-optimal query execution plans chosen by modern query optimizers. We take a step towards a principled and practical approach to performing cardinality estimation without making the independence assumption. By carefully using concepts from the field of graphical models, we are able to factor the joint probability distribution over all the attributes in the database into small, usually two-dimensional distributions, without a significant loss in estimation accuracy. We show how to efficiently construct such a graphical model from the database using only two-way join queries, and we show how to perform selectivity estimation in a highly efficient manner. We integrate our algorithms into the PostgreSQL DBMS. Experimental results indicate that estimation errors can be greatly reduced, leading to orders of magnitude more efficient query execution plans in many cases. Optimization time is kept in the range of tens of milliseconds, making this a practical approach for industrial-strength query optimizers.     

Real-time deformable models for surgery simulation: a survey

Simulating the behaviour of elastic objects in real time is one of the current objectives of computer graphics. One of its fields of application lies in virtual reality, mainly in surgery simulation systems. In computer graphics, the models used for the construction of objects with deformable behaviour are known as deformable models. These have two conflicting characteristics: interactivity and motion realism. The different deformable models developed to date have promoted only one of these (usually interactivity) to the detriment of the other (biomechanical realism). In this paper, we present a classification of the different deformable models that have been developed. We present the advantages and disadvantages of each one. Finally, we make a comparison of deformable models and perform an evaluation of the state of the art and the future of deformable models.     

Markov models for offline handwriting recognition: a survey

Since their first inception more than half a century ago, automatic reading systems have evolved substantially, thereby showing impressive performance on machine-printed text. The recognition of handwriting can, however, still be considered an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic offline handwriting recognition. However, so far, no standard procedures for building Markov-model-based recognizers could be established though trends toward unified approaches can be identified. It is therefore the goal of this survey to provide a comprehensive overview of the application of Markov models in the research field of offline handwriting recognition, covering both the widely used hidden Markov models and the less complex Markov-chain or n-gram models. First, we will introduce the typical architecture of a Markov-model-based offline handwriting recognition system and make the reader familiar with the essential theoretical concepts behind Markovian models. Then, we will give a thorough review of the solutions proposed in the literature for the open problems how to apply Markov-model-based approaches to automatic offline handwriting recognition.     

Interventions and belief change in possibilistic graphical models

Causality and belief change play an important role in many applications. This paper focuses on the main issues of causality and interventions in possibilistic graphical models. We show that interventions, which are very useful for representing causal relations between events, can be naturally viewed as a belief change process. In particular, interventions can be handled using a possibilistic counterpart of Jeffrey's rule of conditioning under uncertain inputs. This paper also addresses new issues that are arisen in the revision of graphical models when handling interventions. We first argue that the order in which observations and interventions are introduced is very important. Then we show that in order to correctly handle sequences of observations and interventions, one needs to change the structure of possibilistic networks. Lastly, an efficient procedure for revising possibilistic causal trees is provided.     

Matching natural language patterns to graphical system models

We outline an approach to parsing based on system modelling. The underlying assumption, which determines the limits of the approach, is that a narrative natural language text constitutes a symbolic model of the system described, written for the purpose of communicating static and/or dynamic system aspects.     

Optimising dynamic graphical models for video content analysis

A key problem in video content analysis using dynamic graphical models is to learn a suitable model structure given observed visual data. We propose a completed likelihood AIC (CL-AIC) scoring function for solving the problem. CL-AIC differs from existing scoring functions in that it aims to optimise explicitly both the explanation and prediction capabilities of a model simultaneously. CL-AIC is derived as a general scoring function suitable for both static and dynamic graphical models with hidden variables. In particular, we formulate CL-AIC for determining the number of hidden states for a hidden Markov model (HMM) and the topology of a dynamically multi-linked HMM (DML-HMM). The effectiveness of CL-AIC on learning the optimal structure of a dynamic graphical model especially given sparse and noisy visual date is shown through comparative experiments against existing scoring functions including Bayesian information criterion (BIC), Akaike’s information criterion (AIC), integrated completed likelihood (ICL), and variational Bayesian (VB). We demonstrate that CL-AIC is superior to the other scoring functions in building dynamic graphical models for solving two challenging problems in video content analysis: (1) content based surveillance video segmentation and (2) discovering causal/temporal relationships among visual events for group activity modelling.     

A graphical characterization of lattice conditional independence models

Lattice conditional independence (LCI) models for multivariate normal data recently have been introduced for the analysis of non-monotone missing data patterns and of nonnested dependent linear regression models (≡ seemingly unrelated regressions). It is shown here that the class of LCI models coincides with a subclass of the class of graphical Markov models determined by acyclic digraphs (ADGs), namely, the subclass of transitive ADG models. An explicit graph-theoretic characterization of those ADGs that are Markov equivalent to some transitive ADG is obtained. This characterization allows one to determine whether a specific ADG D is Markov equivalent to some transitive ADG, hence to some LCI model, in polynomial time, without an exhaustive search of the (possibly superexponentially large) equivalence class [D]. These results do not require the existence or positivity of joint densities. This revised version was published online in June 2006 with corrections to the Cover Date.     

AND/OR search spaces for graphical models

The paper introduces an AND/OR search space perspective for graphical models that include probabilistic networks (directed or undirected) and constraint networks. In contrast to the traditional (OR) search space view, the AND/OR search tree displays some of the independencies present in the graphical model explicitly and may sometimes reduce the search space exponentially. Indeed, most algorithmic advances in search-based constraint processing and probabilistic inference can be viewed as searching an AND/OR search tree or graph. Familiar parameters such as the depth of a spanning tree, treewidth and pathwidth are shown to play a key role in characterizing the effect of AND/OR search graphs vs. the traditional OR search graphs. We compare memory intensive AND/OR graph search with inference methods, and place various existing algorithms within the AND/OR search space.     

Weighted heuristic anytime search: new schemes for optimization over graphical models

Weighted heuristic search (best-first or depth-first) refers to search with a heuristic function multiplied by a constant w [31]. The paper shows, for the first time, that for optimization queries in graphical models the weighted heuristic best-first and weighted heuristic depth-first branch and bound search schemes are competitive energy-minimization anytime optimization algorithms. Weighted heuristic best-first schemes were investigated for path-finding tasks. However, their potential for graphical models was ignored, possibly because of their memory costs and because the alternative depth-first branch and bound seemed very appropriate for bounded depth. The weighted heuristic depth-first search has not been studied for graphical models. We report on a significant empirical evaluation, demonstrating the potential of both weighted heuristic best-first search and weighted heuristic depth-first branch and bound algorithms as approximation anytime schemes (that have sub-optimality bounds) and compare against one of the best depth-first branch and bound solvers to date.