Extending a blackboard architecture for approximate processing

Approximate processing is an approach to real-time AI problem-solving systems in domains where there are a range of acceptable answers in terms of certainty, accuracy, and completeness. Such a system needs to evaluate the current situation, make time predictions about the likelihood of achieving current objectives, monitor the processing and pursuit of those objectives, and if necessary, choose new objectives and associated processing strategies that are achievable in the available time. In this approach, the system is performingsatisficing problem-solving, in that it is attempting to generate the best possible solutions within available time and computational resource constraints.Previously published work (Lesser, Pavlin and Durfee 1988) has dealt with this approach to real-time; however, an important aspect was not fully developed: the problem solver must be very flexible in its ability to represent and efficiently implement a variety of processing strategies. Extensions to the blackboard model of problem solving that facilitate approximate processing are demonstrated for the task of knowledge-based signal interpretation. This is accomplished by extending the blackboard model of problem solving to include data, knowledge, and control approximations. With minimal overhead, the problem solver dynamically responds to the current situation by altering its operators and state space abstraction to produce a range of acceptable answers. Initial experiments with this approach show promising results in both providing a range of processing algorithms and in controlling this dynamic system with low overhead.This work was partly supported by the Office of Naval Research under a University Research Initiative grant, number N00014-86-K-0764, NSF-CER contract DCR-8500332, and ONR contract N00014-89-J-1877.     

Random Forests with ensemble of feature spaces

Random Forests receive much attention from researchers because of their excellent performance. As Breiman suggested, the performance of Random Forests depends on the strength of the weak learners in the forests and the diversity among them. However, in the literature, many researchers only considered pre-processing of the data or post-processing of the Random Forests models. In this paper, we propose a new method to increase the diversity of each tree in the forests and thereby improve the overall accuracy. During the training process of each individual tree in the forest, different rotation spaces are concatenated into a higher space at the root node. Then the best split is exhaustively searched within this higher space. The location where the best split lies decides which rotation method to be used for all subsequent nodes. The performance of the proposed method here is evaluated on 42 benchmark data sets from various research fields and compared with the standard Random Forests. The results show that the proposed method improves the performance of the Random Forests in most cases.     

Random feature weights for decision tree ensemble construction

This paper proposes a method for constructing ensembles of decision trees, random feature weights (RFW). The method is similar to Random Forest, they are methods that introduce randomness in the construction method of the decision trees. In Random Forest only a random subset of attributes are considered for each node, but RFW considers all of them. The source of randomness is a weight associated with each attribute. All the nodes in a tree use the same set of random weights but different from the set of weights in other trees. So, the importance given to the attributes will be different in each tree and that will differentiate their construction. The method is compared to Bagging, Random Forest, Random-Subspaces, AdaBoost and MultiBoost, obtaining favourable results for the proposed method, especially when using noisy data sets. RFW can be combined with these methods. Generally, the combination of RFW with other method produces better results than the combined methods. Kappa-error diagrams and Kappa-error movement diagrams are used to analyse the relationship between the accuracies of the base classifiers and their diversity.     

Forest fire forecasting using ensemble learning approaches

Neural Computing and Applications - Frequent and intense forest fires have posed severe challenges to forest management in many countries worldwide. Since human experts may overlook important...     

Evaluation of Random Forest and Adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery

Detailed land use/land cover classification at ecotope level is important for environmental evaluation. In this study, we investigate the possibility of using airborne hyperspectral imagery for the classification of ecotopes. In particular, we assess two tree-based ensemble classification algorithms: Adaboost and Random Forest, based on standard classification accuracy, training time and classification stability. Our results show that Adaboost and Random Forest attain almost the same overall accuracy (close to 70%) with less than 1% difference, and both outperform a neural network classifier (63.7%). Random Forest, however, is faster in training and more stable. Both ensemble classifiers are considered effective in dealing with hyperspectral data. Furthermore, two feature selection methods, the out-of-bag strategy and a wrapper approach feature subset selection using the best-first search method are applied. A majority of bands chosen by both methods concentrate between 1.4 and 1.8 μm at the early shortwave infrared region. Our band subset analyses also include the 22 optimal bands between 0.4 and 2.5 μm suggested in Thenkabail et al. [Thenkabail, P.S., Enclona, E.A., Ashton, M.S., and Van Der Meer, B. (2004). Accuracy assessments of hyperspectral waveband performance for vegetation analysis applications. Remote Sensing of Environment, 91, 354-376.] due to similarity of the target classes. All of the three band subsets considered in this study work well with both classifiers as in most cases the overall accuracy dropped only by less than 1%. A subset of 53 bands is created by combining all feature subsets and comparing to using the entire set the overall accuracy is the same with Adaboost, and with Random Forest, a 0.2% improvement. The strategy to use a basket of band selection methods works better. Ecotopes belonging to the tree classes are in general classified better than the grass classes. Small adaptations of the classification scheme are recommended to improve the applicability of remote sensing method for detailed ecotope mapping.     

决策树与随机森林

决策树算法是数据挖掘领域的一个研究热点。决策树代表的是对象属性与对象值之间的一种映射关系,以树状结构表现,在实际中应用广泛。笔者首先介绍了信息论,重点阐述了三种典型的决策树分类算法原理,并分析了不同算法的优缺点,最后介绍了基于决策树的随机森林算法及其在机器学习中的作用。     

Extending type systems in a library: Type-safe XML processing in C++

Type systems built directly into the compiler or interpreter of a programming language cannot be easily extended to keep track of run-time invariants of new abstractions. Yet, programming with domain-specific abstractions could benefit from additional static checking. This paper presents library techniques for extending the type system of C++ to support domain-specific abstractions. The main contribution is a programmable “subtype” relation. As a demonstration of the techniques, we implement a type system for defining type qualifiers in C++, as well as a type system for the XML processing language, capable of, e.g., statically guaranteeing that a program only produces valid XML documents according to a given XML schema.     

Measurement of information processing load and visual load on a dynamic information processing task

A dynamic visual information processing task was designed to investigate time-based and intensity-based factors on an operator's information processing load as measured by reaction time, pupil diameter, and eye movement parameters. The time-based factor was manipulated by the target rate and scanning rate while the intensity-based factor was manipulated by the difference between a simple reaction task and a physical matching (choice reaction) task. Nine participants tracked the scanning line at two different scanning rates and were required to respond to two designated targets presented singly at two different temporal frequencies. The results indicated that task difficulty (the intensity-based factor) had a significant effect on the reaction time. Target rate and scanning rate were integrated as one time-based factor in terms of three sweeping angles. The time-based factor was found to have a significant effect on the fixation time, saccade amplitude, fixation frequency, eye movement speed, reaction time and hit rate. No interaction effect was found between time-based and intensity-based factors. The time pressure (defined by the time required divided by the time available) based on a model human processor was positively related to scanning rate, target rate and task difficulty. It was found to be the most objective and reliable if time required can be reliably predicted based on a predictive model approach.     

Modeling Random Fuzzy Renewal Reward Processes

This short paper discusses the modeling of random fuzzy renewal reward processes in which the interarrival times and rewards are represented by nonnegative random fuzzy variables. Based on random fuzzy theory, a random fuzzy variable denotes a measurable function from a credibility space to a collection of random variables. Under this setting, the long-run expected reward per unit time is addressed and the theorem on random fuzzy renewal rewards is established. The utility of this research is demonstrated with a realistic application case.      

Towards a standard arabic information processing vocabulary

This paper is concerned with the development of a standard Arabic information processing vocabulary. It has been considered that the development of such a vocabulary does not only involve the translation of information processing terms into Arabic, but it also includes the provision of standard definitions for them. The paper establishes the necessary ground for the required plan by reviewing the special features of the information processing terms, and how they have been dealt with by various institutions and individuals, both internationally, and within the Arab world. The proposed plan consists of several tasks, and considers the necessity of obtaining pan-Arab recognition of the target vocabulary, and the importance of cooperation with international efforts.     

A variant of Rotation Forest for constructing ensemble classifiers

Rotation Forest, an effective ensemble classifier generation technique, works by using principal component analysis (PCA) to rotate the original feature axes so that different training sets for learning base classifiers can be formed. This paper presents a variant of Rotation Forest, which can be viewed as a combination of Bagging and Rotation Forest. Bagging is used here to inject more randomness into Rotation Forest in order to increase the diversity among the ensemble membership. The experiments conducted with 33 benchmark classification data sets available from the UCI repository, among which a classification tree is adopted as the base learning algorithm, demonstrate that the proposed method generally produces ensemble classifiers with lower error than Bagging, AdaBoost and Rotation Forest. The bias–variance analysis of error performance shows that the proposed method improves the prediction error of a single classifier by reducing much more variance term than the other considered ensemble procedures. Furthermore, the results computed on the data sets with artificial classification noise indicate that the new method is more robust to noise and kappa-error diagrams are employed to investigate the diversity–accuracy patterns of the ensemble classifiers.     

Efficient processing of nested Fuzzy SQL queries in a fuzzydatabase

In a fuzzy relational database where a relation is a fuzzy set of tuples and ill-known data are represented by possibility distributions, nested fuzzy queries can be expressed in the Fuzzy SQL language. Although it provides a very convenient way for users to express complex queries, a nested fuzzy query may be very inefficient to process with the naive evaluation method based on its semantics. In conventional databases, nested queries are unnested to improve the efficiency of their evaluation. In this paper, we extend the unnesting techniques to process several types of nested fuzzy queries. An extended merge-join is used to evaluate the unnested fuzzy queries. As shown by both theoretical analysis and experimental results, the unnesting techniques with the extended merge-join significantly improve the performance of evaluating nested fuzzy queries     

Random forests-based extreme learning machine ensemble for multi-regime time series prediction

Accurate and timely predicting values of performance parameters are currently strongly needed for important complex equipment in engineering. In time series prediction, two problems are urgent to be solved. One problem is how to achieve the accuracy, stability and efficiency together, and the other is how to handle time series with multiple regimes. To solve these two problems, random forests-based extreme learning machine ensemble model and a novel multi-regime approach are proposed respectively, and these two approaches can be integrated to achieve better performance. First, the extreme learning machine (ELM) is used in the proposed model because of its efficiency. Then the regularized ELM and ensemble learning strategy are used to improve generalization performance and prediction accuracy. The bootstrap sampling technique is used to generate training sample sets for multiple base-level ELM models, and then the random forests (RF) model is used as the combiner to aggregate these ELM models to achieve more accurate and stable performance. Next, based on the specific properties of turbofan engine time series, a multi-regime approach is proposed to handle it. Regimes are first separated, then the proposed RF-based ELM ensemble model is used to learn models of all regimes, individually, and last, all the learned regime models are aggregated to predict performance parameter at the future timestamp. The proposed RF-based ELM ensemble model and multi-regime approaches are evaluated by using NN3 time series and NASA turbofan engine time series, and then the proposed model is applied to the exhaust gas temperature prediction of CFM engine. The results demonstrate that the proposed RF-based ELM ensemble model and multi-regime approach can be accurate, stable and efficient in predicting multi-regime time series, and it can be robust against overfitting.     

Exploring the diversity in cluster ensemble generation: Random sampling and random projection

Cluster ensemble first generates a large library of different clustering solutions and then combines them into a more accurate consensus clustering. It is commonly accepted that for cluster ensemble to work well the member partitions should be different from each other, and meanwhile the quality of each partition should remain at an acceptable level. Many different strategies have been used to generate different base partitions for cluster ensemble. Similar to ensemble classification, many studies have been focusing on generating different partitions of the original dataset, i.e., clustering on different subsets (e.g., obtained using random sampling) or clustering in different feature spaces (e.g., obtained using random projection). However, little attention has been paid to the diversity and quality of the partitions generated using these two approaches. In this paper, we propose a novel cluster generation method based on random sampling, which uses the nearest neighbor method to fill the category information of the missing samples (abbreviated as RS-NN). We evaluate its performance in comparison with k-means ensemble, a typical random projection method (Random Feature Subset, abbreviated as FS), and another random sampling method (Random Sampling based on Nearest Centroid, abbreviated as RS-NC). Experimental results indicate that the FS method always generates more diverse partitions while RS-NC method generates high-quality partitions. Our proposed method, RS-NN, generates base partitions with a good balance between the quality and the diversity and achieves significant improvement over alternative methods. Furthermore, to introduce more diversity, we propose a dual random sampling method which combines RS-NN and FS methods. The proposed method can achieve higher diversity with good quality on most datasets.