Community detection in complex networks: Multi-objective discrete backtracking search optimization algorithm with decomposition

Community detection is believed to be a very important tool for understanding both the structure and function of complex networks, and has been intensively investigated in recent years. Community detection can be considered as a multi-objective optimization problem and the nature-inspired optimization techniques have shown promising results in dealing with this problem. In this study, we present a novel multi-objective discrete backtracking search optimization algorithm with decomposition for community detection in complex networks. First, we present a discrete variant of the backtracking search optimization algorithm (DBSA) where the updating rules of individuals are redesigned based on the network topology. Then, a novel multi-objective discrete method (MODBSA/D) based on the proposed discrete variant DBSA is first proposed to minimize two objective functions in terms of Negative Ratio Association (NRA) and Ratio Cut (RC) of community detection problems. Finally, the proposed algorithm is tested on some real-world networks to evaluate its performance. The results clearly show that MODBSA/D has effective and promising performance for dealing with community detection in complex networks.     

基于遗传神经网络的相似重复记录检测方法

为了有效解决数据清洗领域中相似重复记录的检测问题,提出了一种基于遗传神经网络的相似重复记录检测方法.该方法计算两条记录对应字段间的相似度,构建基于神经网络的检测模型,利用遗传算法对网络模型的权值进行优化,使用遗传神经网络组合多个字段上的相似度来检测相似重复记录.在不同领域数据集上的测试结果表明,该方法能够提高相似重复记录检测的准确率和检测精度.     

一种无线传感器网络故障节点检测算法

提出一种无线传感器网络故障节点的检测方法,无需事件或模型假设,通过识别节点序列中违反排名的节点找到故障节点.算法对实际应用中的噪声环境和子序列估计问题分别提出了相应的解决方法.仿真实验表明:在不同的网络设置下,漏检率和误检率均较低,算法具有良好的性能.     

A cusum change-point detection algorithm for non-stationary sequences with application to data network surveillance

We adapt the classic cusum change-point detection algorithm to handle non-stationary sequences that are typical with network surveillance applications. The proposed algorithm uses a defined timeslot structure to take into account time varying distributions, and uses historical samples of observations within each timeslot to facilitate a nonparametric methodology. Our proposed solution includes an on-line screening feature that fully automates the implementation of the algorithm and eliminates the need for manual oversight up until the point where root cause analysis begins.     

A three-phase algorithm for flowshop scheduling with blocking to minimize makespan

This paper proposes a three-phase algorithm (TPA) for the flowshop scheduling problem with blocking (BFSP) to minimize makespan. In the first phase, the blocking nature of BFSP is exploited to develop a priority rule that creates a sequence of jobs. Using this as the initial sequence and a variant of the NEH-insert procedure, the second phase generates an approximate solution to the problem. Then, utilizing a modified simulated annealing algorithm incorporated with a local search procedure, the schedule generated in the second phase is improved in the third phase. A pruning procedure that helps evaluate most solutions without calculating their complete makespan values is introduced in the local search to further reduce the computational time needed to solve the problem. Results of the computational experiments with Taillard's benchmark problem instances show that the proposed TPA algorithm is relatively more effective and efficient in minimizing makespan for the BFSP than the state-of-the-art procedures. Utilizing these results, 53 out of 60 new tighter upper bounds have been found for large-sized Taillard's benchmark problem instances.     

PDDRA: A new pre-fetching based dynamic data replication algorithm in data grids

In recent years, grid technology has had such a fast growth that it has been used in many scientific experiments and research centers. A large number of storage elements and computational resources are combined to generate a grid which gives us shared access to extra computing power. In particular, data grid deals with data intensive applications and provides intensive resources across widely distributed communities. Data replication is an efficient way for distributing replicas among the data grids, making it possible to access similar data in different locations of the data grid. Replication reduces data access time and improves the performance of the system. In this paper, we propose a new dynamic data replication algorithm named PDDRA that optimizes the traditional algorithms. Our proposed algorithm is based on an assumption: members in a VO (Virtual Organization) have similar interests in files. Based on this assumption and also file access history, PDDRA predicts future needs of grid sites and pre-fetches a sequence of files to the requester grid site, so the next time that this site needs a file, it will be locally available. This will considerably reduce access latency, response time and bandwidth consumption. PDDRA consists of three phases: storing file access patterns, requesting a file and performing replication and pre-fetching and replacement. The algorithm was tested using a grid simulator, OptorSim developed by European Data Grid projects. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, effective network usage, total number of replications, hit ratio and percentage of storage filled.     

A fast projection algorithm for sequence data searching

In real life, data often appear in the form of sequences and this form of data is called sequence data. In this paper, a new definition on sequence similarity and a novel algorithm, Projection Algorithm, for sequence data searching are proposed. This algorithm is not required to access every datum in a sequence database. However, it guarantees that no qualified subsequence is falsely rejected. Moreover, the projection algorithm can be extended to match subsequences with different scales. With careful selection of parameters, most of the similar subsequences with different scales can be retrieved. We also show by experiments that the proposed algorithm can outperform the traditional sequential searching algorithm up to 96 times in terms of speed up.     

A data clustering algorithm for stratified data partitioning in artificial neural network

The statistical properties of training, validation and test data play an important role in assuring optimal performance in artificial neural networks (ANNs). Researchers have proposed optimized data partitioning (ODP) and stratified data partitioning (SDP) methods to partition of input data into training, validation and test datasets. ODP methods based on genetic algorithm (GA) are computationally expensive as the random search space can be in the power of twenty or more for an average sized dataset. For SDP methods, clustering algorithms such as self organizing map (SOM) and fuzzy clustering (FC) are used to form strata. It is assumed that data points in any individual stratum are in close statistical agreement. Reported clustering algorithms are designed to form natural clusters. In the case of large multivariate datasets, some of these natural clusters can be big enough such that the furthest data vectors are statistically far away from the mean. Further, these algorithms are computationally expensive as well. We propose a custom design clustering algorithm (CDCA) to overcome these shortcomings. Comparisons are made using three benchmark case studies, one each from classification, function approximation and prediction domains. The proposed CDCA data partitioning method is evaluated in comparison with SOM, FC and GA based data partitioning methods. It is found that the CDCA data partitioning method not only perform well but also reduces the average CPU time.     

A memetic algorithm for the flexible flow line scheduling problem with processor blocking

This paper introduces an efficient memetic algorithm (MA) combined with a novel local search engine, namely, nested variable neighbourhood search (NVNS), to solve the flexible flow line scheduling problem with processor blocking (FFLB) and without intermediate buffers. A flexible flow line consists of several processing stages in series, with or without intermediate buffers, with each stage having one or more identical parallel processors. The line produces a number of different products, and each product must be processed by at most one processor in each stage. To obtain an optimal solution for this type of complex, large-sized problem in reasonable computational time using traditional approaches and optimization tools is extremely difficult. Our proposed MA employs a new representation, operators, and local search method to solve the above-mentioned problem. The computational results obtained in experiments demonstrate the efficiency of the proposed MA, which is significantly superior to the classical genetic algorithm (CGA) under the same conditions when the population size is increased in the CGA.     

一种无线传感器网络目标检测动态阈值算法

介绍了无线传感器网络目标检测基本流程;着重分析了无线传感器网络在实际应用中每个传感器自身存在目标检测范围差异的问题;针对这种情况下的无线传感器网络目标检测问题,提出了一种传感器本地决策阈值的动态算法;最后通过仿真实验,比较了动态阈值算法与其他三种目标检测算法的目标检测误报率。仿真结果表明,提出的动态阈值算法具有较低的检测误报率。     

DSM-FI: an efficient algorithm for mining frequent itemsets in data streams

Online mining of data streams is an important data mining problem with broad applications. However, it is also a difficult problem since the streaming data possess some inherent characteristics. In this paper, we propose a new single-pass algorithm, called DSM-FI (data stream mining for frequent itemsets), for online incremental mining of frequent itemsets over a continuous stream of online transactions. According to the proposed algorithm, each transaction of the stream is projected into a set of sub-transactions, and these sub-transactions are inserted into a new in-memory summary data structure, called SFI-forest (summary frequent itemset forest) for maintaining the set of all frequent itemsets embedded in the transaction data stream generated so far. Finally, the set of all frequent itemsets is determined from the current SFI-forest. Theoretical analysis and experimental studies show that the proposed DSM-FI algorithm uses stable memory, makes only one pass over an online transactional data stream, and outperforms the existing algorithms of one-pass mining of frequent itemsets.

A hybrid data-fusion system using modal data and probabilistic neural network for damage detection

This paper addresses a novel hybrid data-fusion system for damage detection by integrating the data fusion technique, probabilistic neural network (PNN) models and measured modal data. The hybrid system proposed consists of three models, i.e. a feature-level fusion model, a decision-level fusion model and a single PNN classifier model without data fusion. Underlying this system is the idea that we can choose any of these models for damage detection under different circumstances, i.e. the feature-level model is preferable to other models when enormous data are made available through multi-sensors, whereas the confidence level for each of multi-sensors must be determined (as a prerequisite) before the adoption of the decision-level model, and lastly, the single model is applicable only when data collected is somehow limited as in the cases when few sensors have been installed or are known to be functioning properly. The hybrid system is suitable for damage detection and identification of a complex structure, especially when a huge volume of measured data, often with uncertainties, are involved, such as the data available from a large-scale structural health monitoring system. The numerical simulations conducted by applying the proposed system to detect both single- and multi-damage patterns of a 7-storey steel frame show that the hybrid data-fusion system cannot only reliably identify damage with different noise levels, but also have excellent anti-noise capability and robustness.     

Information fusion for anomaly detection with the dendritic cell algorithm

Dendritic cells are antigen presenting cells that provide a vital link between the innate and adaptive immune system, providing the initial detection of pathogenic invaders. Research into this family of cells has revealed that they perform information fusion which directs immune responses. We have derived a dendritic cell algorithm based on the functionality of these cells, by modelling the biological signals and differentiation pathways to build a control mechanism for an artificial immune system. We present algorithmic details in addition to experimental results, when the algorithm was applied to anomaly detection for the detection of port scans. The results show the dendritic cell algorithm is successful at detecting port scans.     

WeVoS-ViSOM: An ensemble summarization algorithm for enhanced data visualization

This study presents a novel version of the Visualization Induced Self-Organizing Map based on the application of a new fusion algorithm for summarizing the results of an ensemble of topology-preserving mapping models. The algorithm is referred to as Weighted Voting Superposition (WeVoS). Its main feature is the preservation of the topology of the map, in order to obtain the most accurate possible visualization of the data sets under study. To do so, a weighted voting process between the units of the maps in the ensemble takes place, in order to determine the characteristics of the units of the resulting map. Several different quality measures are applied to this novel neural architecture known as WeVoS-ViSOM and the results are analyzed, so as to present a thorough study of its capabilities. To complete the study, it has also been compared with the well-know SOM and its fusion version, with the WeVoS-SOM and with two other previously devised fusion Fusion by Euclidean Distance and Fusion by Voronoi Polygon Similarity—based on the analysis of the same quality measures in order to present a complete analysis of its capabilities. All three summarization methods were applied to three widely used data sets from the UCI Repository. A rigorous performance analysis clearly demonstrates that the novel fusion algorithm outperforms the other single and summarization methods in terms of data sets visualization.     

Negative selection algorithm with constant detectors for anomaly detection

In the paper, two novel negative selection algorithms (NSAs) were proposed: FB-NSA and FFB-NSA. FB-NSA has two types of detectors: constant-sized detector (CFB-NSA) and variable-sized detector (VFB-NSA). The detectors of traditional NSA are generated randomly. Even for the same training samples, the position, size, and quantity of the detectors generated in each time are different. In order to eliminate the effect of training times on detectors, in the proposed approaches, detectors are generated in non-random ways. To determine the performances of the approaches, the experiments on 2-dimensional synthetic datasets, Iris dataset and ball bearing fault data were performed. Results show that FB-NSA and FFB-NSA outperforms the other anomaly detection methods in most cases. Besides, CFB-NSA can detect the abnormal degree of mechanical equipment. To determine the performances of CFB-NSA, the experiments on ball bearing fault data were performed. Results show that the abnormal degree based on the CFB-NSA can be used to diagnose the different fault types with the same fault degree, and the same fault type with the different fault degree.     

数据缺失下学习贝叶斯网络的E-GSA算法

针对数据缺失条件下构建贝叶斯网络难度大的问题,研究了贝叶斯结构学习算法,提出了将条件独立性检验和评分-搜索相结合的算法.采用改进的混合算法对训练数据初始化,建立相应的初始网络,对已经拟合了训练数据信息的初始网络用遗传模拟退火算法进行训练以找到最佳的网络结构.给出了算法实施的具体步骤且通过实验验证了算法性能,并将实验结果与其他典型的算法进行比较,表明了算法具有更优的学习效果.     

A k-populations algorithm for clustering categorical data

In this paper, the conventional k-modes-type algorithms for clustering categorical data are extended by representing the clusters of categorical data with k-populations instead of the hard-type centroids used in the conventional algorithms. Use of a population-based centroid representation makes it possible to preserve the uncertainty inherent in data sets as long as possible before actual decisions are made. The k-populations algorithm was found to give markedly better clustering results through various experiments.     

MMR: An algorithm for clustering categorical data using Rough Set Theory

A variety of cluster analysis techniques exist to group objects having similar characteristics. However, the implementation of many of these techniques is challenging due to the fact that much of the data contained in today’s databases is categorical in nature. While there have been recent advances in algorithms for clustering categorical data, some are unable to handle uncertainty in the clustering process while others have stability issues. This research proposes a new algorithm for clustering categorical data, termed Min–Min-Roughness (MMR), based on Rough Set Theory (RST), which has the ability to handle the uncertainty in the clustering process.     

A cloud detection algorithm for AATSR data, optimized for daytime observations in Canada

To extract information about the Earth's surface from Earth Observation data, a key processing step is the separation of pixels representing clear-sky observations of land or water surfaces from observations substantially influenced by clouds. This paper presents an algorithm used for this purpose specifically for data from the AATSR sensor on ENVISAT. The algorithm is based on the structure of the SPARC cloud detection scheme developed at CCRS for AVHRR data, then modified, calibrated and validated for AATSR data. It uses a series of weighted tests to calculate per-pixel cloud presence probability, and also produces an estimate of cloud top height and a cloud shadow flag. Algorithm parameters have been optimized for daytime use in Canada, and evaluation shows good performance with a mean daytime kappa coefficient of 0.76 for the ‘cloud’/‘clear’ classification when compared to independent validation data. Performance is independent of season, and is a dramatic improvement over the existing AATSR L1B cloud flag for Canada. The algorithm will be used at CCRS for processing AATSR data, and will form the basis of similar processing for data from the SLSTR sensors on Sentinel-3.