Fuzzy classifications using fuzzy inference networks

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.     

Video Behaviour Mining Using a Dynamic Topic Model

This paper addresses the problem of fully automated mining of public space video data, a highly desirable capability under contemporary commercial and security considerations. This task is especially challenging due to the complexity of the object behaviors to be profiled, the difficulty of analysis under the visual occlusions and ambiguities common in public space video, and the computational challenge of doing so in real-time. We address these issues by introducing a new dynamic topic model, termed a Markov Clustering Topic Model (MCTM). The MCTM builds on existing dynamic Bayesian network models and Bayesian topic models, and overcomes their drawbacks on sensitivity, robustness and efficiency. Specifically, our model profiles complex dynamic scenes by robustly clustering visual events into activities and these activities into global behaviours with temporal dynamics. A Gibbs sampler is derived for offline learning with unlabeled training data and a new approximation to online Bayesian inference is formulated to enable dynamic scene understanding and behaviour mining in new video data online in real-time. The strength of this model is demonstrated by unsupervised learning of dynamic scene models for four complex and crowded public scenes, and successful mining of behaviors and detection of salient events in each.     

基于多源异构数据融合的网络安全态势评估体系

针对基于单点网络数据很难准确地检测网络恶意活动且无法有效地分析网络状况的问题,本文通过引入多源异构数据融合策略,借鉴层次化网络分析思想,构建出包含流量探测模块、属性提炼模块、决策引擎模块、多源融合模块、态势评估模块等五大模块的网络安全态势评估体系。评估体系以BP神经网络为决策引擎分析各数据源的数据,使用指数加权D-S证据理论融合各决策引擎的输出结果,并基于层次化网络威胁评估方法评估网络威胁状况。实验结果表明:不同探测器探测到的数据对于识别不同类型攻击的优势不同;多源融合技术进一步将识别攻击类型的准确率提升到88.7%;层次化网络威胁评估方法能够有效地评估网络威胁状况。     

Stochastic reasoning for structural pattern recognition: An example from image-based UAV navigation

This paper reports on the statistical embedding of a structural pattern recognition system into the autonomous navigation of an unmanned aerial vehicle (UAV). A rule-based system is used for the recognition of visual landmarks such as bridges in aerial views. In principle, rule-based systems can be designed and coded with no training data at hand, but a sound interpretation and utilization of the achieved results needs statistical inference and representative data sets of sufficient coverages. Flying a UAV with an experimental system is expensive, risky, and legally questionable. Therefore, we chose a virtual globe as a camera simulator providing arbitrary amounts of training and test data. The expected positions of landmarks in the aerial views are modeled by mixture models representing inliers, outliers, and intermediate forms which stem from similar structures appearing frequently in the vicinity of landmarks. The parameters of the corresponding likelihood functions are estimated by the Expectation–Maximization method. Using these estimates, we carry out tests and compare the results for heuristic, pessimistic, optimistic, and Bayesian decision rationales. This performance evaluation reveals the superiority of the Bayesian approach.     

Context-prediction performance by a dynamic Bayesian network: Emphasis on location prediction in ubiquitous decision support environment

Ubiquitous decision support systems require more intelligent mechanism in which more timely and accurate decision support is available. However, conventional context-aware systems, which have been popular in the ubiquitous decision support systems field, cannot provide such agile and proactive decision support. To fill this research void, this paper proposes a new concept of context prediction mechanism by which the ubiquitous decision support devices are able to predict users’ future contexts in advance, and provide more timely and proactive decision support that users would be satisfied much more. Especially, location prediction is useful because ubiquitous decision support systems could dynamically adapt their decision support contents for a user based on a user’s future location. In this sense, as an alternative for the inference engine mechanism to be used in the ubiquitous decision support systems capable of context-prediction, we propose an inductive approach to recognizing a user’s location by learning a dynamic Bayesian network model. The dynamic Bayesian network model has been evaluated with a set of contextual data from undergraduate students. The evaluation result suggests that a dynamic Bayesian network model offers significant predictive power in the location prediction. Besides, we found that the dynamic Bayesian network model has a great potential for the future types of ubiquitous decision support systems.     

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.     

Fuzzy inference in rule-based real-time control

In this paper research into the application of ‘expert system’-like inference mechanisms in the field of fuzzy control is adressed. Using techniques from the area of rule-based expert systems, a more flexible way of design and modification is presented of new and existing fuzzy systems for modelling and control. In comparison with ‘normal’ applications of fuzzy inference, the ćompositional rule of inference is replaced by a fuzzy inference engine. General applicability of the fuzzy inference engine is made possible by its general character as a fuzzy expert system shell. Succesful implementations in simulation and realtime control environments show the flexibility and usefullness of the described fuzzy inference engine.     

Collaborative information acquisition for data-driven decisions

Data-driven predictive models are routinely used by government agencies and industry to improve the efficiency of their decision-making. In many cases, agencies acquire training data over time, incurring both direct and opportunity costs. Active learning can be used to acquire particularly informative training data that improve learning cost-effectively. However, when multiple models are used to inform decisions, prior work on active learning has significant limitations: either it improves the accuracy of predictive models without regard to how accuracy affects decision making or it addresses only decisions informed by a single predictive model. We propose that decisions informed by multiple models warrant a new kind of Collaborative Information Acquisition (CIA) policy that allows multiple learners to reason collaboratively about informative acquisitions. This paper focuses on tax audit decisions, which affect a vital revenue source for governments worldwide. Because audits are costly to conduct, active learning policies can help identify particularly informative audits to improve future decisions. However, existing active learning models are poorly suited to audit decisions, because audits are best informed by multiple predictive models. We develop a CIA policy to improve the decisions the models inform, and we demonstrate that CIA can substantially increase sales tax revenues. We also demonstrate that the CIA policy can improve decisions to target directly individuals in a donation campaign. Finally, we discuss and demonstrate the risks for decision making of the naive use of existing active learning policies.     

HELIC-II: Legal reasoning system on the parallel inference machine

This paper presents HELIC-II, a legal reasoning system on the parallel inference machine. HELIC-II draws legal conclusions for a given case by referring to a statutory law (legal rules) and judicial precedents (old cases). This system consists of two inference engines. The rule-based engine draws legal consequences logically by using legal rules. The case-based engine generates legal concepts by referencing similar old cases. These engines complementally draw all possible conclusions, and output them in the form of inference trees. Users can use these trees as material to construct arguments in a legal suit. HELIC-II is implemented on the parallel inference machine, and it can draw conclusions quickly by parallel inference. As an example, a legal inference system for the Penal Code is introduced, and the effectiveness of the legal reasoning and parallel inference model is shown.     

Designing high-level decision making systems based on fuzzy if–then rules for a point-to-point car racing game

In this paper, fuzzy rule-based systems are applied to a point-to-point car racing game. In the point-to-point car racing game, two car agents compete with each other for taking waypoints. There are three waypoints in the car racing field, each of which is assigned a number that indicates the order to take. The control process of car agents is modeled as a non-holonomic system where there are two input variables (acceleration and steering) for controlling the position, angle and velocity of the car agents. Fuzzy rule-based systems are used to make a high-level decision where the target waypoint to take is determined. Since a fuzzy rule-based system for the high-level decision making is generated in the manner of supervised learning, a set of training patterns should be given for the construction of the fuzzy rule-based systems. In this paper we examine two methods to obtain such a set of training patterns. We also examine two representations of input vectors for the fuzzy rule-based systems. We discuss the effect of obtained training patterns and the input representation on the performance of the fuzzy rule-based systems. After discussing and analyzing the experimental results, we present an adaptive framework of fuzzy rule-based systems. The performance of adaptive fuzzy rule-based systems is then examined based on the results of their non-adaptive version. A series of computational experiments are performed to show the learning ability of the adaptive fuzzy rule-based systems.     

专家系统中基于粗集的知识获取、更新与推理

知识获取、知识更新和不确定性推理是设计专家系统的重要方面。根据粗集理论,提出了一种专家系统的结构模型,该系统在规则获取的基础上,利用系统运行的实例增量式地更新知识库中的规则及其参数,以改善系统的性能,利用知识库中的规则及数量参数进行不确定性推理,得出结论的可信度。     

Multiple Bayesian discriminant functions for high-dimensional massive data classification

The presence of complex distributions of samples concealed in high-dimensional, massive sample-size data challenges all of the current classification methods for data mining. Samples within a class usually do not uniformly fill a certain (sub)space but are individually concentrated in certain regions of diverse feature subspaces, revealing the class dispersion. Current classifiers applied to such complex data inherently suffer from either high complexity or weak classification ability, due to the imbalance between flexibility and generalization ability of the discriminant functions used by these classifiers. To address this concern, we propose a novel representation of discriminant functions in Bayesian inference, which allows multiple Bayesian decision boundaries per class, each in its individual subspace. For this purpose, we design a learning algorithm that incorporates the naive Bayes and feature weighting approaches into structural risk minimization to learn multiple Bayesian discriminant functions for each class, thus combining the simplicity and effectiveness of naive Bayes and the benefits of feature weighting in handling high-dimensional data. The proposed learning scheme affords a recursive algorithm for exploring class density distribution for Bayesian estimation, and an automated approach for selecting powerful discriminant functions while keeping the complexity of the classifier low. Experimental results on real-world data characterized by millions of samples and features demonstrate the promising performance of our approach.     

An adaptive rule-based classifier for mining big biological data

In this paper, we introduce a new adaptive rule-based classifier for multi-class classification of biological data, where several problems of classifying biological data are addressed: overfitting, noisy instances and class-imbalance data. It is well known that rules are interesting way for representing data in a human interpretable way. The proposed rule-based classifier combines the random subspace and boosting approaches with ensemble of decision trees to construct a set of classification rules without involving global optimisation. The classifier considers random subspace approach to avoid overfitting, boosting approach for classifying noisy instances and ensemble of decision trees to deal with class-imbalance problem. The classifier uses two popular classification techniques: decision tree and k-nearest-neighbor algorithms. Decision trees are used for evolving classification rules from the training data, while k-nearest-neighbor is used for analysing the misclassified instances and removing vagueness between the contradictory rules. It considers a series of k iterations to develop a set of classification rules from the training data and pays more attention to the misclassified instances in the next iteration by giving it a boosting flavour. This paper particularly focuses to come up with an optimal ensemble classifier that will help for improving the prediction accuracy of DNA variant identification and classification task. The performance of proposed classifier is tested with compared to well-approved existing machine learning and data mining algorithms on genomic data (148 Exome data sets) of Brugada syndrome and 10 real benchmark life sciences data sets from the UCI (University of California, Irvine) machine learning repository. The experimental results indicate that the proposed classifier has exemplary classification accuracy on different types of biological data. Overall, the proposed classifier offers good prediction accuracy to new DNA variants classification where noisy and misclassified variants are optimised to increase test performance.     

Transparent decision support using statistical reasoning and fuzzy inference

A framework for the development of a decision support system (DSS) that exhibits uncommonly transparent rule-based inference logic is introduced. A DSS is constructed by marrying a statistically based fuzzy inference system (FIS) with a user interface, allowing drill-down exploration of the underlying statistical support, providing transparent access to both the rule-based inference as well as the underlying statistical basis for the rules. The FIS is constructed through a "pattern discovery" based analysis of training data. Such an analysis yields a rule base characterized by simple explanations for any rule or data division in the extracted knowledge base. The reliability of a fuzzy inference is well predicted by a confidence measure that determines the probability of a correct suggestion by examination of values produced within the inference calculation. The combination of these components provides a means of constructing decision support systems that exhibit a degree of transparency beyond that commonly observed in supervised-learning-based methods. A prototype DSS is analyzed in terms of its workflow and usability, outlining the insight derived through use of the framework. This is demonstrated by considering a simple synthetic data example and a more interesting real-world example application with the goal of characterizing patients with respect to risk of heart disease. Specific input data samples and corresponding output suggestions created by the system are presented and discussed. The means by which the suggestions made by the system may be used in a larger decision context is evaluated.     

Fault detection and diagnostic method of diesel engine by combining rule-based algorithm and BNs/BPNNs

The stable operation of diesel engine is critical to the normal production of the industry, and the prevention, monitoring, and identification of faults are of great significance. At present, the fault research on diesel engines still has some defects, such as only few types of faults diagnosis are identified, the accuracy of fault diagnosis is still low, and fault identification is located at a fixed speed. A novel fault detection and diagnostic method of diesel engine by combining rule-based algorithm and Bayesian networks (BNs) or Back Propagation neural networks (BPNNs) is proposed. The signals are processed by wavelet threshold denoising and ensemble empirical mode decomposition. The signal-sensitive feature values are extracted from the decomposed intrinsic mode function. Seven faults are roughly identified using rule-based algorithm and finely identified using BNs or BPNNs. Results show the proposed fault diagnosis method has a good diagnostic performance for a wide range of rotation speeds when the training data for BNs and BPNNs are from fixed speeds. In addition, the influences of the layers of decomposed signals, sensor noise and external excitation interference on the fault diagnostic performance are also researched.     

Deep Generative Crowdsourcing Learning with Worker Correlation Utilization

Traditional supervised learning requires the groundtruth labels for the training data, which can be difficult to collect in many cases. In contrast, crowdsourcing learning collects noisy annotations from multiple non-expert workers and infers the latent true labels through some aggregation approach. In this paper, we notice that existing deep crowdsourcing work does not sufficiently model worker correlations, which is, however, shown to be helpful for learning by previous non-deep learning approaches. We propose a deep generative crowdsourcing learning approach to incorporate the strengths of Deep Neural Networks (DNNs) and exploit worker correlations. The model comprises a DNN classifier as a prior and an annotation generation process. A mixture model of workers'' capabilities within each class is introduced into the annotation generation process for worker correlation modeling. For adaptive trade-off between model complexity and data fitting, we implement fully Bayesian inference. Based on the natural-gradient stochastic variational inference techniques developed for the Structured Variational AutoEncoder (SVAE), we combine variational message passing for conjugate parameters and stochastic gradient descent for DNN parameters into a unified framework for efficient end-to-end optimization. Experimental results on 22 real crowdsourcing datasets demonstrate the effectiveness of the proposed approach.     

分布式深度学习隐私与安全攻击研究进展与挑战

不同于集中式深度学习模式,分布式深度学习摆脱了模型训练过程中数据必须中心化的限制,实现了数据的本地操作,允许各方参与者在不交换数据的情况下进行协作,显著降低了用户隐私泄露风险,从技术层面可以打破数据孤岛,显著提升深度学习的效果,能够广泛应用于智慧医疗、智慧金融、智慧零售和智慧交通等领域.但生成对抗式网络攻击、成员推理攻击和后门攻击等典型攻击揭露了分布式深度学习依然存在严重隐私漏洞和安全威胁.首先对比分析了联合学习、联邦学习和分割学习3种主流的分布式深度学习模式特征及其存在的核心问题.其次,从隐私攻击角度,全面阐述了分布式深度学习所面临的各类隐私攻击,并归纳和分析了现有隐私攻击防御手段.同时,从安全攻击角度,深入剖析了数据投毒攻击、对抗样本攻击和后门攻击3种安全攻击方法的攻击过程和内在安全威胁,并从敌手能力、防御原理和防御效果等方面对现有安全攻击防御技术进行了度量.最后,从隐私与安全攻击角度,对分布式深度学习未来的研究方向进行了讨论和展望.     

基于规则的受限委派框架

委派(delegation)允许特权在主体间传播,是信任管理系统实现跨域授权的核心机制,但不加限制的委派可导致特权扩散,削弱了信息系统的安全性.现有信任管理系统的委派机制缺乏有效的特权传播控制能力,委派机制的安全性也有待于严格的分析和证明.文中提出了基于角色的受限委派模型RCDM,能够支持灵活的特权委派策略,并采用一种范围约束(scope constraint)结构控制特权传播的深度范围和广度范围.面向RCDM提出一种基于规则的满足性验证算法C3A,基于逻辑程序语义理论分析了C3A算法关于RCDM的可靠性和完备性问题,从理论上证明了RCDM的安全性和可用性.     

Power-aware anomaly detection in smartphones: An analysis of on-platform versus externalized operation

Many security problems in smartphones and other smart devices are approached from an anomaly detection perspective in which the main goal reduces to identifying anomalous activity patterns. Since machine learning algorithms are generally used to build such detectors, one major challenge is adapting these techniques to battery-powered devices. Many recent works simply assume that on-platform detection is prohibitive and suggest using offloaded (i.e., cloud-based) engines. Such a strategy seeks to save battery life by exchanging computation and communication costs, but it still remains unclear whether this is optimal or not in all circumstances. In this paper, we evaluate different strategies for offloading certain functional tasks in machine learning based detection systems. Our experimental results confirm the intuition that outsourced computation is clearly the best option in terms of power consumption, outweighing on-platform strategies in, essentially, all practical scenarios. Our findings also point out noticeable differences among different machine learning algorithms, and we provide separate consumption models for functional blocks (data preprocessing, training, test, and communications) that can be used to obtain power consumption estimates and compare detectors.