Two-group classification with high-dimensional correlated data: A factor model approach

A class of linear classification rules, specifically designed for high-dimensional problems, is proposed. The new rules are based on Gaussian factor models and are able to incorporate successfully the information contained in the sample correlations. Asymptotic results, that allow the number of variables to grow faster than the number of observations, demonstrate that the worst possible expected error rate of the proposed rules converges to the error of the optimal Bayes rule when the postulated model is true, and to a slightly larger constant when this model is a reasonable approximation to the data generating process. Numerical comparisons suggest that, when combined with appropriate variable selection strategies, rules derived from one-factor models perform comparably, or better, than the most successful extant alternatives under the conditions they were designed for. The proposed methods are implemented as an R package named HiDimDA, available from the CRAN repository.     

A note on “A new method for ranking discovered rules from data mining by DEA”, and a full ranking approach

In a recent paper by Toloo et al. [Toloo, M., Sohrabi, B., & Nalchigar, S. (2009). A new method for ranking discovered rules from data mining by DEA. Expert Systems with Applications, 36, 8503–8508], they proposed a new integrated data envelopment analysis model to find most efficient association rule in data mining. Then, utilizing this model, an algorithm is developed for ranking association rules by considering multiple criteria. In this paper, we show that their model only selects one efficient association rule by chance and is totally depended on the solution method or software is used for solving the problem. In addition, it is shown that their proposed algorithm can only rank efficient rules randomly and will fail to rank inefficient DMUs. We also refer to some other drawbacks in that paper and propose another approach to set up a full ranking of the association rules. A numerical example illustrates some contents of the paper.     

Applying constraint satisfaction approach to solve product configuration problems with cardinality-based configuration rules

In this paper, the product configuration problems that are characterized by cardinality-based configuration rules are dealt with. Novel configuration rules including FI and EI rules are presented to clarify the semantics of inclusion rules when cardinalities and hierarchies of products are encountered. Then, a configuration graph is proposed to visualize structural rules and configuration rules in product configuration problem. An encoding approach is elaborated to transform the configuration graph as a CSP (Constraint Satisfaction Problem). As a consequence, existing CSP solver, i.e. JCL (Java Constraint Library), is employed to implement the configuration system for product configuration problem with cardinality-related configuration rules. A case study of a bus configuration is used throughout this paper to illustrate the effectiveness of the presented approach.     

Rule base reduction for knowledge-based fuzzy controllers with application to a vacuum cleaner

A rule base reduction and tuning algorithm is proposed as a design tool for the knowledge-based fuzzy control of a vacuum cleaner. Given a set of expert-based control rules in a fuzzy rule base structure, proposed algorithm computes the inconsistencies and redundancies in the overall rule set based on a newly proposed measure of equality of the individual fuzzy sets. An inconsistency and redundancy measure is proposed and computed for each rule in the rule base. Then the rules with high inconsistency and redundancy levels are removed from the fuzzy rule base without affecting the overall performance of the controller. The algorithm is successfully tested experimentally for the control of a commercial household vacuum cleaner. Experimental results demonstrate the effective use of the proposed algorithm.     

Systemic approach to fuzzy logic formalization for approximate reasoning

L.A. Zadeh, E.H. Mamdani, M. Mizumoto, et al., R.A. Aliev and A. Tserkovny have proposed methods for fuzzy reasoning in which antecedents and consequents involve fuzzy conditional propositions of the form “If x is A then y is B”, with A and B being fuzzy concepts (fuzzy sets). A formulation of fuzzy antecedent/consequent chains is one of the most important topics within a wide spectrum of problems in fuzzy sets in general and approximate reasoning, in particular. From the analysis of relevant research it becomes clear that for this purpose, a so-called fuzzy conditional inference rules comes as a viable alternative. In this study, we present a systemic approach toward fuzzy logic formalization for approximate reasoning. For this reason, we put together some comparative analysis of fuzzy reasoning methods in which antecedents contain a conditional proposition with fuzzy concepts and which are based on implication operators present in various types of fuzzy logic. We also show a process of a formation of the fuzzy logic regarded as an algebraic system closed under all its operations. We examine statistical characteristics of the proposed fuzzy logic. As the matter of practical interest, we construct a set of fuzzy conditional inference rules on the basis of the proposed fuzzy logic. Continuity and stability features of the formalized rules are investigated.     

Domain-driven KDD for mining functionally novel rules and linking disjoint medical hypotheses

IntroductionAn important quality of association rules is novelty. However, evaluating rule novelty is AI-hard and has been a serious challenge for most data mining systems.ObjectiveIn this paper, we introduce functional novelty, a new non-pairwise approach to evaluating rule novelty. A functionally novel rule is interesting as it suggests previously unknown relations between user hypotheses.MethodsWe developed a novel domain-driven KDD framework for discovering functionally novel association rules. Association rules were mined from cardiovascular data sets. At post-processing, domain knowledge-compliant rules were discovered by applying semantic-based filtering based on UMLS ontology. Their knowledge compliance scores were computed against medical knowledge in Pubmed literature. A cardiologist explored possible relationships between several pairs of unknown hypotheses. The functional novelty of each rule was computed based on its likelihood to mediate these relationships.ResultsHighly interesting rules were successfully discovered. For instance, common rules such as diabetes mellitus?coronary arteriosclerosis was functionally novel as it mediated a rare association between von Willebrand factor and intracardiac thrombus.ConclusionThe proposed post-mining domain-driven rule evaluation technique and measures proved to be useful for estimating candidate functionally novel rules with the results validated by a cardiologist.     

A simulation study on the performance of pickup-dispatching rules for multiple-load AGVs

In this paper, the pickup-dispatching problem of multiple-load AGVs (automated guided vehicles) is studied. This problem is defined in the multiple-load control process proposed by Ho and Chien [Ho, Y. C., & Chien, S. H. (2004). A simulation study on the performance of delivery-dispatching rules for multiple-load AGVs. In E. Kozan (Ed.), Proceedings of abstracts and papers (On CD-ROM) of the 5th Asia-Pacific industrial engineering and management systems conference and the 7th Asia-Pacific division meeting of the international foundation of production research (pp. 18.1.1–18.1.15). Brisbane: APIEMS]. Their control process identifies four problems faced by a multiple-load AGV. These problems are task-determination, delivery-dispatching, pickup-dispatching and load-selection. This paper focuses on the third problem. For this problem, nine pickup-dispatching rules are proposed and studied. The first, second and fourth problems are not the main focus of this study, thus only one task-determination rule, one delivery-dispatching rule and two load-selection rules are adopted for them. The objective of this study is twofold. First, to understand the performance of the proposed rules in different performance measures, e.g., the system’s throughput, the mean flow time of parts (MFTP) and the mean tardiness of parts (MTP). Second, the effects that the proposed rules have on each other’s performance are investigated. Computer simulations are used to achieve these objectives. The experimental results reveal a rule that dispatches vehicles to the machine with the greatest output queue length is the best in all performance measures. Also, distance-based or due-time-based rules do not perform as well as queue-based rules. It is also found that the performance of pickup-dispatching rules is affected by different load-selections rules.     

A comprehensive framework for modeling set-based business rules during conceptual database design

Business rules are the basis of any organization. From an information systems perspective, these business rules function as constraints on a database helping ensure that the structure and content of the real world—sometimes referred to as miniworld—is accurately incorporated into the database. It is important to elicit these rules during the analysis and design stage, since the captured rules are the basis for subsequent development of a business constraints repository. We present a taxonomy for set-based business rules, and describe an overarching framework for modeling rules that constrain the cardinality of sets. The proposed framework results in various types constraints, i.e., attribute, class, participation, projection, co-occurrence, appearance and overlapping, on a semantic model that supports abstractions like classification, generalization/specialization, aggregation and association. We formally define the syntax of our proposed framework in Backus-Naur Form and explicate the semantics using first-order logic. We describe partial ordering in the constraints and define the concept of metaconstraints, which can be used for automatic constraint consistency checking during the design stage itself. We demonstrate the practicality of our approach with a case study and show how our approach to modeling business rules seamlessly integrates into existing database design methodology. Via our proposed framework, we show how explicitly capturing data semantics will help bridge the semantic gap between the real world and its representation in an information system.     

Further remark on P systems with active membranes and two polarizations

P systems (or membrane systems) are a class of distributed parallel computing devices of a biochemical type. In this paper, some restrictions on the general form of the developing rules are considered, under which it is still possible to solve NP-complete problems. We present an algorithm for deterministically deciding SAT in linear time by P systems with active membranes using two polarizations and rules of restricted versions of types (a), (c), (e). The result obtained in this paper answered an open problem proposed by Alhazov and Freund in the aspect of computing efficiency.     

Mining fuzzy β-certain and β-possible rules from quantitative data based on the variable precision rough-set model

The rough-set theory proposed by Pawlak, has been widely used in dealing with data classification problems. The original rough-set model is, however, quite sensitive to noisy data. Ziarko thus proposed the variable precision rough-set model to deal with noisy data and uncertain information. This model allowed for some degree of uncertainty and misclassification in the mining process. Conventionally, the mining algorithms based on the rough-set theory identify the relationships among data using crisp attribute values; however, data with quantitative values are commonly seen in real-world applications. This paper thus deals with the problem of producing a set of fuzzy certain and fuzzy possible rules from quantitative data with a predefined tolerance degree of uncertainty and misclassification. A new method, which combines the variable precision rough-set model and the fuzzy set theory, is thus proposed to solve this problem. It first transforms each quantitative value into a fuzzy set of linguistic terms using membership functions and then calculates the fuzzy β-lower and the fuzzy β-upper approximations. The certain and possible rules are then generated based on these fuzzy approximations. These rules can then be used to classify unknown objects. The paper thus extends the existing rough-set mining approaches to process quantitative data with tolerance of noise and uncertainty.     

ETARM: an efficient top-k association rule mining algorithm

Mining association rules plays an important role in data mining and knowledge discovery since it can reveal strong associations between items in databases. Nevertheless, an important problem with traditional association rule mining methods is that they can generate a huge amount of association rules depending on how parameters are set. However, users are often only interested in finding the strongest rules, and do not want to go through a large amount of rules or wait for these rules to be generated. To address those needs, algorithms have been proposed to mine the top-k association rules in databases, where users can directly set a parameter k to obtain the k most frequent rules. However, a major issue with these techniques is that they remain very costly in terms of execution time and memory. To address this issue, this paper presents a novel algorithm named ETARM (Efficient Top-k Association Rule Miner) to efficiently find the complete set of top-k association rules. The proposed algorithm integrates two novel candidate pruning properties to more effectively reduce the search space. These properties are applied during the candidate selection process to identify items that should not be used to expand a rule based on its confidence, to reduce the number of candidates. An extensive experimental evaluation on six standard benchmark datasets show that the proposed approach outperforms the state-of-the-art TopKRules algorithm both in terms of runtime and memory usage.     

On discovery of soft associations with “most” fuzzy quantifier for item promotion applications

In item promotion applications, there is a strong need for tools that can help to unlock the hidden profit within each individual customer’s transaction history. Discovering association patterns based on the data mining technique is helpful for this purpose. However, the conventional association mining approach, while generating “strong” association rules, cannot detect potential profit-building opportunities that can be exposed by “soft” association rules, which recommend items with looser but significant enough associations. This paper proposes a novel mining method that automatically detects hidden profit-building opportunities through discovering soft associations among items from historical transactions. Specifically, this paper proposes a relaxation method of association mining with a new support measurement, called soft support, that can be used for mining soft association patterns expressed with the “most” fuzzy quantifier. In addition, a novel measure for validating the soft-associated rules is proposed based on the estimated possibility of a conditioned quantified fuzzy event. The new measure is shown to be effective by comparison with several existing measures. A new association mining algorithm based on modification of the FT-Tree algorithm is proposed to accommodate this new support measure. Finally, the mining algorithm is applied to several data sets to investigate its effectiveness in finding soft patterns and content recommendation.     

Machine learning approach to model sport training

The aim of this study was to use a machine learning approach combining fuzzy modeling with an immune algorithm to model sport training, in particular swimming. A proposed algorithm mines the available data and delivers the results in a form of a set of fuzzy rules “IF (fuzzy conditions) THEN (class)”. Fuzzy logic is a powerful method to cope with continuous data, to overcome problem of overlapping class definitions, and to improve the rule comprehensibility. Sport training is modeled at the level of microcycle and training unit by 12 independent attributes. The data was collected in two months (February-March 2008), among swimmers from swimming sections in Wroc?aw, Poland. The swimmers had minimum of 7 years of training and reached the II class level in swimming classification from 2005 to 2008. The goal of the performed experiments was to find the rules answering the question - how does the training unit influence swimmer’s feelings while being in water the next day? The fuzzy rules were inferred for two different scales of the class to be predicted. The effectiveness of the learned set of rules reached 68.66%. The performance, in terms of classification accuracy, of the proposed approach was compared with traditional classifier schemes. The accuracy of the result of compared methods is significantly lower than the accuracy of fuzzy rules obtained by a method presented in this study (paired t-test, P < 0.05).     

Linguistic fuzzy model identification based on PSO with different length of particles

To generate the structure and parameters of fuzzy rule base automatically, a particle swarm optimization algorithm with different length of particles (DLPPSO) is proposed in the paper. The main finding of the proposed approach is that the structure and parameters of a fuzzy rule base can be generated automatically by the proposed PSO. In this method, the best fitness (f_gbest) and the number (N_gbest) of active rules of the best particle in current generation, the best fitness (f_pbesti) which ith particle has achieved so far and the number (N_pbesti) of active rules of it when the best position emerged are utilized to determine the active rules of ith particle in each generation. To increase the diversity of structure, mutation operator is used to change the number of active rules for particles. Compared with some other PSOs with different length of particles, the algorithm has good adaptive performance. To indicate the effectiveness of the give algorithm, a nonlinear function and two time series are used in the simulation experiments. Simulation results demonstrate that the proposed method can approximate the nonlinear function and forecast the time series efficiently.     

P systems with minimal parallelism

A current research topic in membrane computing is to find more realistic P systems from a biological point of view, and one target in this respect is to relax the condition of using the rules in a maximally parallel way. We contribute in this paper to this issue by considering the minimal parallelism of using the rules: if at least a rule from a set of rules associated with a membrane or a region can be used, then at least one rule from that membrane or region must be used, without any other restriction (e.g., more rules can be used, but we do not care how many). Weak as it might look, this minimal parallelism still leads to universality. We first prove this for the case of symport/antiport rules. The result is obtained both for generating and accepting P systems, in the latter case also for systems working deterministically. Then, we consider P systems with active membranes, and again the usual results are obtained: universality and the possibility to solve NP-complete problems in polynomial time (by trading space for time).     

Inductive discovery of laws using monotonic rules

We are considering knowledge discovery from data describing a piece of real or abstract world. The patterns being induced put in evidence some laws hidden in the data. The most natural representation of patterns-laws is by “if..., then...” decision rules relating some conditions with some decisions. The same representation of patterns is used in multi-attribute classification, thus the data searched for discovery of these patterns can be seen as classification data. We adopt the classification perspective to present an original methodology of inducing general laws from data and representing them by so-called monotonic decision rules. Monotonicity concerns relationships between values of condition and decision attributes, e.g. the greater the mass (condition attribute), the greater the gravity (decision attribute), which is a specific feature of decision rules discovered from data using the Dominance-based Rough Set Approach (DRSA). While in DRSA one has to suppose a priori the presence or absence of positive or negative monotonicity relationships which hold in the whole evaluation space, in this paper, we show that DRSA can be adapted to discover rules from any kind of input classification data, exhibiting monotonicity relationships which are unknown a priori and hold in some parts of the evaluation space only. This requires a proper non-invasive transformation of the classification data, permitting representation of both positive and negative monotonicity relationships that are to be discovered by the proposed methodology. Reported results of a computational experiment confirm that the proposed methodology leads to decision rules whose predictive ability is similar to the best classification predictors. It has, however, a unique advantage over all competitors because the monotonic decision rules can be read as laws characterizing the analyzed phenomena in terms of easily understandable “if..., then...” decision rules, while other predictor models have no such straightforward interpretation.     

Generalized composite integration rules in the presence of a singularity

A composite integration rule results from applying a basic integration rule exact for constants and with all abscissas in the integration interval to each ofn equal subintervals of the interval of integration. A sequence of such composite rules converges to the interval for all Riemann-integrable functions and for functions with an endpoint singularity which are dominated by a monotonic improperly integrable function. These rules are generalized to allow a partition into unequal subintervals or different basic rules in each subinterval. In each of these situations, restrictions must be placed on the basic rule or rules or on the partition to ensure convergence in the singular case. Such restrictions also ensure the convergence of a modified rule in the case of an interior singularity.