Defect Prediction Using Akaike and Bayesian Information Criterion

Data available in software engineering for many applications contains variability and it is not possible to say which variable helps in the process of the prediction. Most of the work present in software defect prediction is focused on the selection of best prediction techniques. For this purpose, deep learning and ensemble models have shown promising results. In contrast, there are very few researches that deals with cleaning the training data and selection of best parameter values from the data. Sometimes data available for training the models have high variability and this variability may cause a decrease in model accuracy. To deal with this problem we used the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) for selection of the best variables to train the model. A simple ANN model with one input, one output and two hidden layers was used for the training instead of a very deep and complex model. AIC and BIC values are calculated and combination for minimum AIC and BIC values to be selected for the best model. At first, variables were narrowed down to a smaller number using correlation values. Then subsets for all the possible variable combinations were formed. In the end, an artificial neural network (ANN) model was trained for each subset and the best model was selected on the basis of the smallest AIC and BIC value. It was found that combination of only two variables’ ns and entropy are best for software defect prediction as it gives minimum AIC and BIC values. While, nm and npt is the worst combination and gives maximum AIC and BIC values.     

An information criterion for optimal neural network selection

The choice of an optimal neural network design for a given problem is addressed. A relationship between optimal network design and statistical model identification is described. A derivative of Akaike's information criterion (AIC) is given. This modification yields an information statistic which can be used to objectively select a ;best' network for binary classification problems. The technique can be extended to problems with an arbitrary number of classes.     

Deep feedback GMDH-type neural network and its application to medical image analysis of MRI brain images

The deep feedback group method of data handling (GMDH)-type neural network is applied to the medical image analysis of MRI brain images. In this algorithm, the complexity of the neural network is increased gradually using the feedback loop calculations. The deep neural network architecture is automatically organized so as to fit the complexity of the medical images using the prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The recognition results show that the deep feedback GMDH-type neural network algorithm is useful for the medical image analysis of MRI brain images, because the optimum neural network architectures fitting the complexity of the medical images are automatically organized so as to minimize the prediction error criterion defined as AIC or PSS.     

Deep multi-layered GMDH-type neural network using revised heuristic self-organization and its application to medical image diagnosis of liver cancer

In this study, the deep multi-layered group method of data handling (GMDH)-type neural network algorithm using revised heuristic self-organization method is proposed and applied to medical image diagnosis of liver cancer. The deep GMDH-type neural network can automatically organize the deep neural network architecture which has many hidden layers. The structural parameters such as the number of hidden layers, the number of neurons in hidden layers and useful input variables are automatically selected to minimize prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The architecture of the deep neural network is automatically organized using the revised heuristic self-organization method which is a type of the evolutionary computation. This new neural network algorithm is applied to the medical image diagnosis of the liver cancer and the recognition results are compared with the conventional 3-layered sigmoid function neural network.     

Medical image analysis of abdominal X-ray CT images by deep multi-layered GMDH-type neural network

In this study, a deep multi-layered group method of data handling (GMDH)-type neural network is applied to the medical image analysis of the abdominal X-ray computed tomography (CT) images. The deep neural network architecture which has many hidden layers are automatically organized using the deep multi-layered GMDH-type neural network algorithm so as to minimize the prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The characteristics of the medical images are very complex and therefore the deep neural network architecture is very useful for the medical image diagnosis and medical image recognition. In this study, it is shown that this deep multi-layered GMDH-type neural network is useful for the medical image analysis of abdominal X-ray CT images.     

Neural discriminant analysis

The role of bootstrap is highlighted for nonlinear discriminant analysis using a feedforward neural network model. Statistical techniques are formulated in terms of the principle of the likelihood of a neural-network model when the data consist of ungrouped binary responses and a set of predictor variables. We illustrate that the information criterion based on the bootstrap method is shown to be favorable when selecting the optimum number of hidden units for a neural-network model. In order to summarize the measure of goodness-of-fit, the deviance on fitting a neural-network model to binary response data can be bootstrapped. We also provide the bootstrap estimates of the biases of excess error in a prediction rule constructed by fitting to the training sample in the neural network model. We also propose bootstrap methods for the analysis of residuals in order to identify outliers and examine distributional assumptions in neural-network model fitting. These methods are illustrated through the analyzes of medical diagnostic data.     

MLP neural network models of CMM measuring processes

The objective of this study is to show how a multi-layer perceptron (MLP) neural network can be used to model a CMM measuring process. To date, most MLP-based process models have been established for process mean only. An innovative approach is proposed to model simultaneously the mean and the variation of a CMM process using one integrated MLP architecture. Therefore, the MLP-based model obtained captures not only the process mean but also the process variation information. Selected issues related to neural network training are also discussed. Specifically, the guideline that was proposed by Mirchandani and Cao (1989) for selecting a number of hidden neurons is tested to determine the effects of the number of hidden neurons. The performances of two different learning algorithms - back-propagation with momentum factor (BPM) and the Fletcher-Reeves (FR) algorithm - are studied in terms of CPU time, training error, and generalization error.     

A new look at the statistical model identification

The history of the development of statistical hypothesis testing in time series analysis is reviewed briefly and it is pointed out that the hypothesis testing procedure is not adequately defined as the procedure for statistical model identification. The classical maximum likelihood estimation procedure is reviewed and a new estimate minimum information theoretical criterion (AIC) estimate (MAICE) which is designed for the purpose of statistical identification is introduced. When there are several competing models the MAICE is defined by the model and the maximum likelihood estimates of the parameters which give the minimum of AIC defined by AIC = (-2)log-(maximum likelihood) + 2(number of independently adjusted parameters within the model). MAICE provides a versatile procedure for statistical model identification which is free from the ambiguities inherent in the application of conventional hypothesis testing procedure. The practical utility of MAICE in time series analysis is demonstrated with some numerical examples.     

An optimal neural network and concrete strength modeling

In this paper, a new algorithm which combines the Akaike's information criterion (AIC) with the golden-section optimization technique has been developed for finding the optimal architecture for single-hidden layer feedforward neural networks. The computational experiments on two analytical functions have verified that the modified AIC criterion is in close agreement with the network generalization. It is observed that as long as proximity to global minimum solution is found for each configuration of the network, the AIC function of the networks over the whole domain is unimodal. Thus, it is suitable for the golden-section search method, that is, very effective and computationally time-saving, especially for large size or complex problems. The proposed optimization algorithm is applied to the modeling of the concrete strength under triaxial stresses.     

Structure identification of generalized adaptive neuro-fuzzy inference systems

This paper presents a method to identify the structure of generalized adaptive neuro-fuzzy inference systems (GANFISs). The structure of GANFIS consists of a number of generalized radial basis function (GRBF) units. The radial basis functions are irregularly distributed in the form of hyper-patches in the input-output space. The minimum number of GRBF units is selected based on a heuristic using the fuzzy curve. For structure identification, a new criterion called structure identification criterion (SIC) is proposed. SIC deals with a trade off between performance and computational complexity of the GANFIS model. The computational complexity of gradient descent learning is formulated based on simulation study. Three methods of initialization of GANFIS, viz., fuzzy curve, fuzzy C-means in x/spl times/y space and modified mountain clustering have been compared in terms of cluster validity measure, Akaike's information criterion (AIC) and the proposed SIC.     

Application of statistical information criteria for optimal fuzzymodel construction

Theoretical studies have shown that fuzzy models are capable of approximating any continuous function on a compact domain to any degree of accuracy. However, constructing a good fuzzy model requires finding a good tradeoff between fitting the training data and keeping the model simple. A simpler model is not only easily understood, but also less likely to overfit the training data. Even though heuristic approaches to explore such a tradeoff for fuzzy modeling have been developed, few principled approaches exist in the literature due to the lack of a well-defined optimality criterion. In this paper, we propose several information theoretic optimality criteria for fuzzy models construction by extending three statistical information criteria: 1) the Akaike information criterion [AIC] (1974); 2) the Bhansali-Downham information criterion [BDIC] (1977); and 3) the information criterion of Schwarz (1978) and Rissanen (1978) [SRIC]. We then describe a principled approach to explore the fitness-complexity tradeoff using these optimality criteria together with a fuzzy model reduction technique based on the singular value decomposition (SVD). The role of these optimality criteria in fuzzy modeling is discussed and their practical applicability is illustrated using a nonlinear system modeling example     

Information methods for model selection in linear mixed effects models with application to HCV data

In this paper, we derive a small sample Akaike information criterion, based on the maximized loglikelihood, and a small sample information criterion based on the maximized restricted loglikelihood in the linear mixed effects model when the covariance matrix of the random effects is known. Small sample corrected information criteria are proposed for a special case of linear mixed effects models, the balanced random-coefficient model, without assuming the random coefficients covariance matrix to be known. A simulation study comparing the derived criteria and several others for model selection in the linear mixed effects models is presented. We illustrate the behavior of the studied information criteria on real data from a study of subjects coinfected with HIV and Hepatitis C virus. Robustness of the criteria, in terms of the error distributed as a mixture of normal distributions, is also studied. Special attention is given to the behavior of the conditional AIC by Vaida and Blanchard (2005). Among the studied criteria, GIC performs best, while cAIC exhibits poor performance. Because of its inferior performance, as demonstrated in this work, we do not recommend its use for model selection in linear mixed effects models.     

BP算法和对称ARCH类模型对股市波动性预测的实证比较

利用我国深圳股票市场的实际数据,建立了相应的BP算法网络预测模型和ARCH(1),GARCH(1,1)预测模型,分别用来对深成指数每个周末收盘价的波动性进行预测．研究表明,BP算法对样本外观测值的上凸曲线拟合得较好,对下凸曲线的拟合效果较差;ARCH(1)和GARCH(1,1)则反之,其预测曲线对样本外观测值的下凸曲线拟合效果都较好,但对上凸曲线的拟合效果都较差．通过采用6种常用的预测误差统计量:平均误差、平均绝对误差、均方根误差、平均绝对比率误差、Akaike信息准则、Baves信息准则对样本外数据的预测结果进行检验,BP算法的预测效果最好,ARCH(1)模型次之,GARcH(1,1)模型偏差．     

Evaluate the number of clusters in finite mixture models with the penalized histogram difference criterion

Aimed at the determination of the number of mixtures for finite mixture models (FMMs), in this work, a new method called the penalized histogram difference criterion (PHDC) is proposed and evaluated with other criteria such as Akaike information criterion (AIC), the minimum message length (MML), the information complexity (ICOMP) and the evidence of data criterion (EDC). The new method, which calculates the penalized histogram difference between the data generated from estimated FMMs and those for modeling purpose, turns out to be better than others for data with complicate mixtures patterns. It is demonstrated in this work that the PHDC can determine the optimal number of clusters of the FMM. Furthermore, the estimated FMMs asymptotically approximate the true model. The utility of the new method is demonstrated through synthetic data sets analysis and the batch-wise comparison of citric acid fermentation processes.     

On an asymptotically optimal adaptive classifier design criterion

A new approach for estimating classification errors is presented. In the model, there are two types of classification error: empirical and generalization error. The first is the error observed over the training samples, and the second is the discrepancy between the error probability and empirical error. In this research, the Vapnik and Chervonenkis dimension (VCdim) is used as a measure for classifier complexity. Based on this complexity measure, an estimate for generalization error is developed. An optimal classifier design criterion (the generalized minimum empirical error criterion (GMEE)) is used. The GMEE criterion consists of two terms: the empirical and the estimate of generalization error. As an application, the criterion is used to design the optimal neural network classifier. A corollary to the Γ optimality of neural-network-based classifiers is proven. Thus, the approach provides a theoretic foundation for the connectionist approach to optimal classifier design. Experimental results to validate this approach     

Heuristics free group method of data handling algorithm of generating optimal partial polynomials with application to air pollution prediction†

In this paper a revised GMDH (Group Method of Data Handling) algorithm is developed in which heuristicsare not required such as dividing the available date. into training data and checking data, predetermining the structure of the partial polynomials, or predetermining the number of intermediate variables. In this algorithm the prediction error criterion, such as PSS (Prediction Sum of Squares) or AIC (Akaike's Information Criterion) evaluated from all the available data, in used as a criterion for generating optimal partial polynomials, for selecting intermediate variables and for stopping the multilayered iterative computation. This heuristics freeGMDH algorithm is applied to non-linear modelling for short-term prediction of air pollution concentration. By using the time series data of SO₂, concentration, the wind velocity and the wind direction in Tokushima; Japan, a suitable model for predicting SO₂concentration at a few hours in advance is developed. The predicted results obtained by the revised GMDH model are compared with the results obtained by a linear regression model, a linear autoregressive model and a. basic GMDH model.     

Comparing measures of model selection for penalized splines in Cox models

This article presents an application and a simulation study of model fit criteria for selecting the optimal degree of smoothness for penalized splines in Cox models. The criteria considered were the Akaike information criterion, the corrected AIC, two formulations of the Bayesian information criterion, and a generalized cross-validation method. The estimated curves selected by the five methods were compared to each other in a study of rectal cancer mortality in autoworkers. In the stimulation study, we estimated the fit of the penalized spline models in six exposure-response scenarios, using the five model fit criteria. The methods were compared on the basis of a mean squared error score and the power and size of hypothesis tests for any effect and for detecting nonlinearity. All comparisons were made across a range in the total sample size and number of cases.     

The AIC Criterion and Symmetrizing the Kullback–Leibler Divergence

The Akaike information criterion (AIC) is a widely used tool for model selection. AIC is derived as an asymptotically unbiased estimator of a function used for ranking candidate models which is a variant of the Kullback-Leibler divergence between the true model and the approximating candidate model. Despite the Kullback-Leibler's computational and theoretical advantages, what can become inconvenient in model selection applications is their lack of symmetry. Simple examples can show that reversing the role of the arguments in the Kullback-Leibler divergence can yield substantially different results. In this paper, three new functions for ranking candidate models are proposed. These functions are constructed by symmetrizing the Kullback-Leibler divergence between the true model and the approximating candidate model. The operations used for symmetrizing are the average, geometric, and harmonic means. It is found that the original AIC criterion is an asymptotically unbiased estimator of these three different functions. Using one of these proposed ranking functions, an example of new bias correction to AIC is derived for univariate linear regression models. A simulation study based on polynomial regression is provided to compare the different proposed ranking functions with AIC and the new derived correction with AIC_c     

Construction of a new BRB based model for time series forecasting

It is important to predict the future behavior of complex systems. Currently there are no effective methods to solve time series forecasting problem by using the quantitative and qualitative information. Therefore, based on belief rule base (BRB), this paper focuses on developing a new model that can deal with the problem. Although it is difficult to obtain accurately and completely quantitative information, some qualitative information can be collected and represented by a BRB. As such, a new BRB based forecasting model is proposed when the quantitative and qualitative information exist simultaneously. The performance of the proposed model depends on the structure and belief degrees of BRB simultaneously. Moreover, the structure is determined by the delay step. In order to obtain the appropriate delay step using the available information, a model selection criterion is defined according to Akaike's information criterion (AIC). Based on the proposed model selection criterion and the optimal algorithm for training the belief degrees, an algorithm for constructing the BRB based forecasting model is developed. Experimental results show that the constructed BRB based forecasting model can not only predict the time series accurately, but also has the appropriate structure.