The aditi deductive database system

Deductive databases generalize relational databases by providing support for recursive views and non-atomic data. Aditi is a deductive system based on the client-server model; it is inherently multi-user and capable of exploiting parallelism on shared-memory multiprocessors. The back-end uses relational technology for efficiency in the management of disk-based data and uses optimization algorithms especially developed for the bottom-up evaluation of logical queries involving recursion. The front-end interacts with the user in a logical language that has more expressive power than relational query languages. We present the structure of Aditi, discuss its components in some detail, and present performance figures.     

Selective sampling for approximate clustering of very large data sets

A key challenge in pattern recognition is how to scale the computational efficiency of clustering algorithms on large data sets. The extension of non‐Euclidean relational fuzzy c‐means (NERF) clustering to very large (VL = unloadable) relational data is called the extended NERF (eNERF) clustering algorithm, which comprises four phases: (i) finding distinguished features that monitor progressive sampling; (ii) progressively sampling from a N × N relational matrix R_N to obtain a n × n sample matrix R_n; (iii) clustering R_n with literal NERF; and (iv) extending the clusters in R_n to the remainder of the relational data. Previously published examples on several fairly small data sets suggest that eNERF is feasible for truly large data sets. However, it seems that phases (i) and (ii), i.e., finding R_n, are not very practical because the sample size n often turns out to be roughly 50% of n, and this over‐sampling defeats the whole purpose of eNERF. In this paper, we examine the performance of the sampling scheme of eNERF with respect to different parameters. We propose a modified sampling scheme for use with eNERF that combines simple random sampling with (parts of) the sampling procedures used by eNERF and a related algorithm sVAT (scalable visual assessment of clustering tendency). We demonstrate that our modified sampling scheme can eliminate over‐sampling of the original progressive sampling scheme, thus enabling the processing of truly VL data. Numerical experiments on a distance matrix of a set of 3,000,000 vectors drawn from a mixture of 5 bivariate normal distributions demonstrate the feasibility and effectiveness of the proposed sampling method. We also find that actually running eNERF on a data set of this size is very costly in terms of computation time. Thus, our results demonstrate that further modification of eNERF, especially the extension stage, will be needed before it is truly practical for VL data. © 2008 Wiley Periodicals, Inc.     

Layered Approach Using Conditional Random Fields for Intrusion Detection

Intrusion detection faces a number of challenges; an intrusion detection system must reliably detect malicious activities in a network and must perform efficiently to cope with the large amount of network traffic. In this paper, we address these two issues of Accuracy and Efficiency using Conditional Random Fields and Layered Approach. We demonstrate that high attack detection accuracy can be achieved by using Conditional Random Fields and high efficiency by implementing the Layered Approach. Experimental results on the benchmark KDD '99 intrusion data set show that our proposed system based on Layered Conditional Random Fields outperforms other well-known methods such as the decision trees and the naive Bayes. The improvement in attack detection accuracy is very high, particularly, for the U2R attacks (34.8 percent improvement) and the R2L attacks (34.5 percent improvement). Statistical Tests also demonstrate higher confidence in detection accuracy for our method. Finally, we show that our system is robust and is able to handle noisy data without compromising performance.     

Patterns Based Classifiers

Data mining is one of the most important areas in the 21 century for its applications are wide ranging. This includes medicine, finance, commerce and engineering, to name a few. Pattern mining is amongst the most important and challenging techniques employed in data mining. Patterns are collections of items which satisfy certain properties. Emerging Patterns are those whose frequencies change significantly from one dataset to another. They represent strong contrast knowledge and have been shown very successful for constructing accurate and robust classifiers. In this paper, we examine various kinds of patterns. We also investigate efficient pattern mining techniques and discuss how to exploit patterns to construct effective classifiers.     

Spectral‐based fault localization using hyperbolic function

Debugging is crucial for producing reliable software. One of the effective bug localization techniques is spectral‐based fault localization. It tries to locate a buggy statement by applying an evaluation metric to program spectra and ranking program components on the basis of the score it computes. Here, we propose a restricted class of “hyperbolic” metrics, with a small number of numeric parameters. This class of functions is based on past theoretical and empirical results. We show that optimization methods such as genetic programming and simulated annealing can reliably discover effective metrics over a wide range of data sets of program spectra. We evaluate the performance for both real programs and model programs with single bugs, multiple bugs, “deterministic” bugs, and nondeterministic bugs and find that the proposed class of metrics performs as well as or better than the previous best‐performing metrics over a broad range of data.     

Traffic analysis zone level crash estimation models based on land use characteristics

The objective of this paper is to develop crash estimation models at traffic analysis zone (TAZ) level as a function of land use characteristics. Crash data and land use data for the City of Charlotte, Mecklenburg County, North Carolina were used to illustrate the development of TAZ level crash estimation models. Negative binomial count models (with log-link) were developed as data was observed to be over-dispersed. Demographic/socio-economic characteristics such as population, the number of household units and employment, traffic indicators such as trip productions and attractions, and, on-network characteristics such as center-lane miles by speed limit were observed to be correlated to land use characteristics, and, hence were not considered in the development of TAZ level crash estimation models. Urban residential commercial, rural district and mixed use district land use variables were observed to be correlated to other land use variables and were also not considered in the development of the models. Results obtained indicate that land use characteristics such as mixed use development, urban residential, single-family residential, multi-family residential, business and, office district are strongly associated and play a statistically significant role in estimating TAZ level crashes. The coefficient for single-family residential area was observed to be negative, indicating a decrease in the number of crashes with an increase in single-family residential area. Models were also developed to estimate these crashes by severity (injury and property damage only crashes). The outcomes can be used in safety conscious planning, land use decisions, long range transportation plans, and, to proactively apply safety treatments in high risk TAZs.     

Atlas: a nested relational database system for text applications

Advanced database applications require facilities such as text indexing, image storage, and the ability to store data with a complex structure. However, these facilities are not usually included in traditional database systems. In this paper we describe Atlas, a nested relational database system that has been designed for text-based applications. The Atlas query language is TQL, an SQL-like query language with text operators. The query language is supported by signature file text indexing techniques, and by a parser that can be configured for different text formats and even some foreign languages. Atlas can also be used to store images and audio