Statistical Models for Data Mining

We review the background to the papers presented in this special issue and give a short introduction to each. We also briefly describe the workshop on Statistical models for data mining, held in Pavia (Italy), in October 2000, where the papers were presented.     

An Experimental Comparison of Model-Based Clustering Methods

We compare the three basic algorithms for model-based clustering on high-dimensional discrete-variable datasets. All three algorithms use the same underlying model: a naive-Bayes model with a hidden root node, also known as a multinomial-mixture model. In the first part of the paper, we perform an experimental comparison between three batch algorithms that learn the parameters of this model: the Expectation–Maximization (EM) algorithm, a winner take all version of the EM algorithm reminiscent of the K-means algorithm, and model-based agglomerative clustering. We find that the EM algorithm significantly outperforms the other methods, and proceed to investigate the effect of various initialization methods on the final solution produced by the EM algorithm. The initializations that we consider are (1) parameters sampled from an uninformative prior, (2) random perturbations of the marginal distribution of the data, and (3) the output of agglomerative clustering. Although the methods are substantially different, they lead to learned models that are similar in quality.     

Accelerating EM for Large Databases

The EM algorithm is a popular method for parameter estimation in a variety of problems involving missing data. However, the EM algorithm often requires significant computational resources and has been dismissed as impractical for large databases. We present two approaches that significantly reduce the computational cost of applying the EM algorithm to databases with a large number of cases, including databases with large dimensionality. Both approaches are based on partial E-steps for which we can use the results of Neal and Hinton (In Jordan, M. (Ed.), Learning in Graphical Models, pp. 355–371. The Netherlands: Kluwer Academic Publishers) to obtain the standard convergence guarantees of EM. The first approach is a version of the incremental EM algorithm, described in Neal and Hinton (1998), which cycles through data cases in blocks. The number of cases in each block dramatically effects the efficiency of the algorithm. We provide amethod for selecting a near optimal block size. The second approach, which we call lazy EM, will, at scheduled iterations, evaluate the significance of each data case and then proceed for several iterations actively using only the significant cases. We demonstrate that both methods can significantly reduce computational costs through their application to high-dimensional real-world and synthetic mixture modeling problems for large databases.     

Therapist and group contact as variables in the behavioral treatment of obesity.

Randomly assigned 29 obese females (average age 48.7 yrs), to 1 of 3 experimental conditions: (a) a "standard" behavioral treatment (SBT) group emphasizing self-management techniques (Ss attended group therapy meetings weekly for 10 wks, then monthly for 6 mo and were given a weight control manual); (b) a group receiving the weight control manual via mail with little professional contact (MMC); and (c) a waiting list control condition. Results reveal a superiority of both treatment conditions over the control condition at posttreatment. SBT Ss did significantly better than MMC Ss at posttreatment but not at the 6-mo follow-up. Weight loss for MMC Ss was minimal. The use of "do-it-yourself" treatment manuals is challenged. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An evaluation of the diagnostic accuracy of Pathfinder.

We present an evaluation of the diagnostic accuracy of Pathfinder, an expert system that assists pathologists with the diagnosis of lymph node diseases. We evaluate two versions of the system using both informal and decision-theoretic metrics of performance. In one version of Pathfinder, we assume incorrectly that all observations are conditionally independent. In the other version, we use a belief network to represent accurately the probabilistic dependencies among the observations. In both versions, we make the assumption--reasonable for this domain--that diseases are mutually exclusive and exhaustive. The results of the study show that (1) it is cost effective to represent probabilistic dependencies among observations in the lymph node domain, and (2) the diagnostic accuracy of the more complex version of Pathfinder is at least as good as that of the Pathfinder expert. In addition, the study illustrates how informal and decision-theoretic metrics for performance complement one another.     

Decision-theoretic case-based reasoning

We describe a decision-theoretic methodology for case-based reasoning in diagnosis and troubleshooting applications. The system utilizes a special-structure Bayesian network to represent diagnostic cases, with nodes representing issues, causes, and symptoms. Dirichlet distributions are assessed at knowledge acquisition time to indicate the strength of relationships between variables. During a diagnosis session, a relevant subnetwork is extracted from a Bayesian-network database that describes a very large number of diagnostic interactions and cases. The constructed network is used to make recommendations regarding possible repairs and additional observations, based on an estimate of expected repair costs. As cases are resolved, observations of issues, causes, symptoms, and the success of repairs are recorded. New variables are added to the database, and the probabilities associated with variables already in the database are updated. In this way, the inferential behavior of system adjusts to the characteristics of the target population of users. We show how these elements work together in a cycle of troubleshooting tasks, and describe some results from a pilot system implementation and deployment     

Probabilistic relevance relations

The intuition behind the construction of Bayesian networks and other graph-based representations of joint probability distributions from expert judgments is based on the assumed relationship between “connectedness” in the graphical model and “relatedness” among the variables involved. We show that several plausible definitions of relatedness do not adhere to such an equivalence. We then provide a definition of probabilistic relatedness that is closely related to connectedness in the graphical model and prove that the two concepts are equivalent whenever the model uses only propositional variables and assuming every combination of value assignment to these variables is feasible. We conjecture that the equivalence established holds also when these restrictions are lifted     

Model-Based Clustering and Visualization of Navigation Patterns on a Web Site

We present a new methodology for exploring and analyzing navigation patterns on a web site. The patterns that can be analyzed consist of sequences of URL categories traversed by users. In our approach, we first partition site users into clusters such that users with similar navigation paths through the site are placed into the same cluster. Then, for each cluster, we display these paths for users within that cluster. The clustering approach we employ is model-based (as opposed to distance-based) and partitions users according to the order in which they request web pages. In particular, we cluster users by learning a mixture of first-order Markov models using the Expectation-Maximization algorithm. The runtime of our algorithm scales linearly with the number of clusters and with the size of the data; and our implementation easily handles hundreds of thousands of user sessions in memory. In the paper, we describe the details of our method and a visualization tool based on it called WebCANVAS. We illustrate the use of our approach on user-traffic data from msnbc.com.     

Pressure release superleak sound modes in He II

The sound modes of He II in a waveguide partially packed with superleak are investigated for the case of a free surface within the waveguide. In the limit of zero vapor density, two propagating modes are found: one a gravity wave whose velocity depends on the superfluid fraction, and the other a temperature wave which is analogous to adiabatic U-tube osculations. With finite vapor the U-tube mode in pure ⁴He mixes strongly with the vapor sound; however, in ³He-⁴He mixtures it is uncoupled from the vapor and is observable. Experimental results are in qualitative agreement with the theory.Alfred P. Sloan Research Fellow.Project supported in part by NSF Contract DMR 76-22306, by ONR Contract N00014-75-C- 0246, and by the Research Corporation.