Individual attribute prior setting methods for naïve Bayesian classifiers

The generalized Dirichlet distribution has been shown to be a more appropriate prior for naïve Bayesian classifiers, because it can release both the negative-correlation and the equal-confidence requirements of the Dirichlet distribution. The previous research did not take the impact of individual attributes on classification accuracy into account, and therefore assumed that all attributes follow the same generalized Dirichlet prior. In this study, the selective naïve Bayes mechanism is employed to choose and rank attributes, and two methods are then proposed to search for the best prior of each single attribute according to the attribute ranks. The experimental results on 18 data sets show that the best approach is to use selective naïve Bayes for filtering and ranking attributes when all of them have Dirichlet priors with Laplace's estimate. After the ranks of the chosen attributes are determined, individual setting is performed to search for the best noninformative generalized Dirichlet prior for each attribute. The selective naïve Bayes is also compared with two representative filters for the feature selection, and the experimental results show that it has the best performance.     

Parameter estimation for generalized Dirichlet distributions from the sample estimates of the first and the second moments of random variables

Generalized Dirichlet distributions have a more flexible covariance structure than Dirichlet distributions, and the computation for the moments of a generalized Dirichlet distribution is still tractable. For situations under which Dirichlet distributions are inappropriate for data analysis, generalized Dirichlet distributions will generally be an applicable alternative. When the expected values and the covariance matrix of random variables can be estimated from available data, this study introduces ways to estimate the parameters of a generalized Dirichlet distribution for analyzing compositional data. Under the assumption that the sample mean of every variable must be considered for parameter estimation, we present methods for choosing the statistics from a sample covariance matrix to construct a generalized Dirichlet distribution. Some rules for removing inappropriate statistics from a sample covariance matrix to speed up the estimation process are also established. An example for Taiwan’s car market is introduced to demonstrate the applicability of the parameter estimation methods.     

Implications of the Dirichlet Assumption for Discretization of Continuous Variables in Naive Bayesian Classifiers

In a naive Bayesian classifier, discrete variables as well as discretized continuous variables are assumed to have Dirichlet priors. This paper describes the implications and applications of this model selection choice. We start by reviewing key properties of Dirichlet distributions. Among these properties, the most important one is perfect aggregation, which allows us to explain why discretization works for a naive Bayesian classifier. Since perfect aggregation holds for Dirichlets, we can explain that in general, discretization can outperform parameter estimation assuming a normal distribution. In addition, we can explain why a wide variety of well-known discretization methods, such as entropy-based, ten-bin, and bin-log l, can perform well with insignificant difference. We designed experiments to verify our explanation using synthesized and real data sets and showed that in addition to well-known methods, a wide variety of discretization methods all perform similarly. Our analysis leads to a lazy discretization method, which discretizes continuous variables according to test data. The Dirichlet assumption implies that lazy methods can perform as well as eager discretization methods. We empirically confirmed this implication and extended the lazy method to classify set-valued and multi-interval data with a naive Bayesian classifier.