A new classifier ensemble methodology based on subspace learning

Different classifiers with different characteristics and methodologies can complement each other and cover their internal weaknesses; so classifier ensemble is an important approach to handle the weakness of single classifier based systems. In this article we explore an automatic and fast function to approximate the accuracy of a given classifier on a typical dataset. Then employing the function, we can convert the ensemble learning to an optimisation problem. So, in this article, the target is to achieve a model to approximate the performance of a predetermined classifier over each arbitrary dataset. According to this model, an optimisation problem is designed and a genetic algorithm is employed as an optimiser to explore the best classifier set in each subspace. The proposed ensemble methodology is called classifier ensemble based on subspace learning (CEBSL). CEBSL is examined on some datasets and it shows considerable improvements.     

A stochastic approach to traffic congestion costs

The real world is a complex dynamic and stochastic environment. This is especially true for the traffic moving daily on our roads. As such, accurate modeling that correctly considers the real-world dynamics and the inherent stochasticity is very important, especially if government will base its road tax decisions on the outcomes of these models. The contemporary traffic prices, if any, however, do not reflect the external congestion costs. In order to induce road users to make the correct decision, marginal external costs should be internalized. To assess these costs, the public sector managers need accurate operational models. We show in this article that using a better representation and characterization of the road traffic, via stochastic queueing models, leads to a more adequate reflection of the congestion costs involved. Using extensive numerical experiments, we show the superiority of the stochastic traffic flow models.     

Network traffic classification based on ensemble learning and co-training

Classification of network traffic is the essential step for many network researches. However,with the rapid evolution of Internet applications the effectiveness of the port-based or payload-based identifi-cation approaches has been greatly diminished in recent years. And many researchers begin to turn their attentions to an alternative machine learning based method. This paper presents a novel machine learning-based classification model,which combines ensemble learning paradigm with co-training tech-niques. Compared to previous approaches,most of which only employed single classifier,multiple clas-sifiers and semi-supervised learning are applied in our method and it mainly helps to overcome three shortcomings:limited flow accuracy rate,weak adaptability and huge demand of labeled training set. In this paper,statistical characteristics of IP flows are extracted from the packet level traces to establish the feature set,then the classification model is created and tested and the empirical results prove its feasibility and effectiveness.     

A general simulator approach to ETC toll traffic congestion

This study proposes a process simulation-based approach using a general simulator to offer solutions for Electronic Toll Collection System (ETC) traffic expressway problems at toll plazas. First, the paper describes an overview of the ETC system in general around the world as well as in Japan, and clarifies the ETC traffic problems that occur on Japanese expressways and hinder the diffusion of the ETC system in Japan. Then the paper describes our approach of process simulation to this issue, using a general simulator to show how the basic model in this study has been built and how its internal procedures are defined. The basic model has been reviewed by a mathematical approach with results shown in the following section. Into this basic model, we applied some actual traffic data obtained from one of the ETC I.C. sites. By using ETC traffic simulations, this study verifies the feasibility of the model. This paper proposes two kinds of solutions to the ETC traffic issue. One solution is gate management, which coordinates ETC/general switching to make the best use of the gates. The key issue here is how to find the best time for gate switching. Another solution is layout redesign, which employs an appropriate new layout for toll plazas. The issue here is to design an effective layout to reduce traffic jams. On-site reviews of either solution represent an unrealistic approach. Therefore, this research utilizes a simulation-based approach to identify these solutions. Using the results of simulation, feasibility of this approach is discussed. Received May 2005 / Revised: January 2006     

The use of learning algorithms in telephone traffic routing—A methodology

A theoretical as well as conceptual framework for the use of learning algorithms in telephone traffic routing is given. The approach is distinctly different from the mathematical programming methods generally used in such cases. Learning algorithms at the network nodes update their strategies for routing traffic on the basis of success or failure in completing calls. The entire system is described as a Markov process and different learning schemes are shown to lead to different flow patterns in the steady state.     

Distributed learning automata based approach to inferring urban structure via traffic flow

Traffic flow can be used as a reference for knowledge generation, which is highly important in urban planning. One of the significant applications of traffic data is decision making about the structure of roads connecting zones of a city. It leads us to an optimal connection between important areas like business centers, shopping malls, construction sites, residential complexes, and other parts of a city which is the motivation of this research. The main question is how to infer the optimal connectivity network considering the current structure of an urban area and time-varying traffic dynamics. Therefore a novel formulation is created in this paper to solve the optimization problem using available data. A proposed algorithm is presented to infer the optimal structure that is a distributed learning automata-based approach. A matrix called estimated optimal connectivity represents the favorite structure and it is optimized utilizing signals about the current system and traffic dynamics from the environment. Two types of data, including synthetic and real-world, are used to show the algorithm’s ability. After many experiments, the algorithm showed capability of optimizing the structure by finding new paths connecting the most correlated areas.

     

Unsupervised feature selection with ensemble learning

Machine Learning - In this paper, we show that the way internal estimates are used to measure variable importance in Random Forests are also applicable to feature selection in unsupervised...     

A survival ensemble of extreme learning machine

Due to the fast learning speed, simplicity of implementation and minimal human intervention, extreme learning machine has received considerable attentions recently, mostly from the machine learning community. Generally, extreme learning machine and its various variants focus on classification and regression problems. Its potential application in analyzing censored time-to-event data is yet to be verified. In this study, we present an extreme learning machine ensemble to model right-censored survival data by combining the Buckley-James transformation and the random forest framework. According to experimental and statistical analysis results, we show that the proposed model outperforms popular survival models such as random survival forest, Cox proportional hazard models on well-known low-dimensional and high-dimensional benchmark datasets in terms of both prediction accuracy and time efficiency.     

Construct support vector machine ensemble to detect traffic incident

This study presents the applicability of support vector machine (SVM) ensemble for traffic incident detection. The SVM has been proposed to solve the problem of traffic incident detection, because it is adapted to produce a nonlinear classifier with maximum generality, and it has exhibited good performance as neural networks. However, the classification result of the practically implemented SVM depends on the choosing of kernel function and parameters. To avoid the burden of choosing kernel functions and tuning the parameters, furthermore, to improve the limited classification performance of the real SVM, and enhance the detection performance, we propose to use the SVM ensembles to detect incident. In addition, we also propose a new aggregation method to combine SVM classifiers based on certainty. Moreover, we proposed a reasonable hybrid performance index (PI) to evaluate the performance of SVM ensemble for detecting incident by combining the common criteria, detection rate (DR), false alarm rate (FAR), mean time to detection (MTTD), and classification rate (CR). Several SVM ensembles have been developed based on bagging, boosting and cross-validation committees with different combining approaches, and the SVM ensemble has been tested on one real data collected at the I-880 Freeway in California. The experimental results show that the SVM ensembles outperform a single SVM based AID in terms of DR, FAR, MTTD, CR and PI. We used one non-parametric test, the Wilcoxon signed ranks test, to make a comparison among six combining schemes. Our proposed combining method performs as well as majority vote and weighted vote. Finally, we also investigated the influence of the size of ensemble on detection performance.     

Recommendation from access logs with ensemble learning

Many recommendation systems find similar users based on a profile of a target user and recommend products that he/she may be interested in. The profile is constructed with his/her purchase histories. However, histories of new customers are not stored and it is difficult to recommend products to them in the same fashion. The problem is called a cold start problem. We propose a recommendation method using access logs instead of purchase histories, because the access logs are gathered more easily than purchase histories and the access logs include much information on their interests. In this study, we construct user’s profiles using product categories browsed by them from their access logs and predict products with Gradient Boosting Decision Tree. In addition, we carry out evaluation experiments using access logs in a real online shop and discuss performance of our proposed method comparing with conventional machine learning and Support Vector Machine (SVM). We confirmed that the proposed method achieved higher precision than SVM over 10 data sets. Especially, under unbalanced data sets, the proposed method is superior to SVM.     

A shape-based similarity measure for time series data with ensemble learning

This paper introduces a shape-based similarity measure, called the angular metric for shape similarity (AMSS), for time series data. Unlike most similarity or dissimilarity measures, AMSS is based not on individual data points of a time series but on vectors equivalently representing it. AMSS treats a time series as a vector sequence to focus on the shape of the data and compares data shapes by employing a variant of cosine similarity. AMSS is, by design, expected to be robust to time and amplitude shifting and scaling, but sensitive to short-term oscillations. To deal with the potential drawback, ensemble learning is adopted, which integrates data smoothing when AMSS is used for classification. Evaluative experiments reveal distinct properties of AMSS and its effectiveness when applied in the ensemble framework as compared to existing measures.     

A compressed sensing approach for efficient ensemble learning

This paper presents a method for improved ensemble learning, by treating the optimization of an ensemble of classifiers as a compressed sensing problem. Ensemble learning methods improve the performance of a learned predictor by integrating a weighted combination of multiple predictive models. Ideally, the number of models needed in the ensemble should be minimized, while optimizing the weights associated with each included model. We solve this problem by treating it as an example of the compressed sensing problem, in which a sparse solution must be reconstructed from an under-determined linear system. Compressed sensing techniques are then employed to find an ensemble which is both small and effective. An additional contribution of this paper, is to present a new performance evaluation method (a new pairwise diversity measurement) called the roulette-wheel kappa-error. This method takes into account the different weightings of the classifiers, and also reduces the total number of pairs of classifiers needed in the kappa-error diagram, by selecting pairs through a roulette-wheel selection method according to the weightings of the classifiers. This approach can greatly improve the clarity and informativeness of the kappa-error diagram, especially when the number of classifiers in the ensemble is large. We use 25 different public data sets to evaluate and compare the performance of compressed sensing ensembles using four different sparse reconstruction algorithms, combined with two different classifier learning algorithms and two different training data manipulation techniques. We also give the comparison experiments of our method against another five state-of-the-art pruning methods. These experiments show that our method produces comparable or better accuracy, while being significantly faster than the compared methods.     

A robust adaptive congestion control strategy for large scale networks with differentiated services traffic

In this paper, a robust decentralized congestion control strategy is developed for a large scale network with Differentiated Services (Diff-Serv) traffic. The network is modeled by a nonlinear fluid flow model corresponding to two classes of traffic, namely the premium traffic and the ordinary traffic. The proposed congestion controller does take into account the associated physical network resource limitations and is shown to be robust to the unknown and time-varying delays. Our proposed decentralized congestion control strategy is developed on the basis of Diff-Serv architecture by utilizing a robust adaptive technique. A Linear Matrix Inequality (LMI) condition is obtained to guarantee the ultimate boundedness of the closed-loop system. Numerical simulation implementations are presented by utilizing the QualNet and Matlab software tools to illustrate the effectiveness and capabilities of our proposed decentralized congestion control strategy.     

The ensemble approach to neural-network learning and generalization

A method is suggested for learning and generalization with a general one-hidden layer feedforward neural network. This scheme encompasses the use of a linear combination of heterogeneous nodes having randomly prescribed parameter values. The learning of the parameters is realized through adaptive stochastic optimization using a generalization data set. The learning of the linear coefficients in the linear combination of nodes is achieved with a linear regression method using data from the training set. One node is learned at a time. The method allows for choosing the proper number of net nodes, and is computationally efficient. The method was tested on mathematical examples and real problems from materials science and technology.     

FuzzyBagging: A novel ensemble of classifiers

In this work, a new method for the creation of classifier ensembles is introduced. The patterns are partitioned into clusters to group together similar patterns, a training set is built using the patterns that belong to a cluster. Each of the new sets is used to train a classifier. We show that the approach here presented, called FuzzyBagging, obtains performance better than Bagging.     

A swarm intelligent method for traffic light scheduling: application to real urban traffic networks

Traffic lights play an important role nowadays for solving complex and serious urban traffic problems. How to optimize the schedule of hundreds of traffic lights has become a challenging and exciting problem. This paper proposes an inner and outer cellular automaton mechanism combined with particle swarm optimization (IOCA-PSO) method to achieve a dynamic and real-time optimization scheduling of urban traffic lights. The IOCA-PSO method includes the inner cellular model (ICM), the outer cellular model (OCM), and the fitness function. Our work can be divided into following parts: (1) Concise basic transition rules and affiliated transition rules are proposed in ICM, which can help the proposed phase cycle planning (PCP) algorithm achieve a globally sophisticated scheduling and offer effective solutions for different traffic problems; (2) Benefited from the combination of cellular automaton (CA) and particle swarm optimization (PSO), the proposed inner and outer cellular PSO (IOPSO) algorithm in OCM offers a strong search ability to find out the optimal timing control; (3) The proposed fitness function can evaluate and conduct the optimization of traffic lights’ scheduling dynamically for different aims by adjusting parameters. Extensive experiments show that, compared with the PSO method, the genetic algorithm method and the RANDOM method in real cases, IOCA-PSO presents distinct improvements under different traffic conditions, which shows a high adaptability of the proposed method in urban traffic network scales under different traffic flow states, intersection numbers, and vehicle numbers.     

A conversational intelligent tutoring system to automatically predict learning styles

This paper proposes a generic methodology and architecture for developing a novel conversational intelligent tutoring system (CITS) called Oscar that leads a tutoring conversation and dynamically predicts and adapts to a student’s learning style. Oscar aims to mimic a human tutor by implicitly modelling the learning style during tutoring, and personalising the tutorial to boost confidence and improve the effectiveness of the learning experience. Learners can intuitively explore and discuss topics in natural language, helping to establish a deeper understanding of the topic. The Oscar CITS methodology and architecture are independent of the learning styles model and tutoring subject domain. Oscar CITS was implemented using the Index of Learning Styles (ILS) model (Felder & Silverman, 1988) to deliver an SQL tutorial. Empirical studies involving real students have validated the prediction of learning styles in a real-world teaching/learning environment. The results showed that all learning styles in the ILS model were successfully predicted from a natural language tutoring conversation, with an accuracy of 61–100%. Participants also found Oscar’s tutoring helpful and achieved an average learning gain of 13%.     

A novel distributed congestion control for bufferless network-on-chip

Bufferless Network-on-Chip (NoC) emerges as an interesting option for NoC design in recent years, which can save considerable router power and area. However, bufferless NoC only works well under low-to-medium load because it becomes more easily congested as message injection rate increases. In this paper, we propose a novel distributed source-throttling congestion control mechanism that relieves the effect of congestion in bufferless NoC under high load, called Cbufferless. The proposed strategy uses a novel congestion detection and control mechanism, computing average deflection rate of routing flit and distributed throttling message injection. Utilizing the new mechanism, the congestion information can be directly obtained inside node, which allows the mechanism to be fully distributed without requiring any transmission of global congestion information among neighbor routers and within a router. Simulation results show that the proposed mechanism improves system throughput by up to $\sim $ 30 and $\sim $ 15.5 %, saves energy consumption by up to $\sim $ 40 and $\sim $ 19 % than that of baseline and injection rate throttling bufferless NoCs, respectively, and keeps lower message latency under congested load when compared.     

A comparative assessment of ensemble learning for credit scoring

Both statistical techniques and Artificial Intelligence (AI) techniques have been explored for credit scoring, an important finance activity. Although there are no consistent conclusions on which ones are better, recent studies suggest combining multiple classifiers, i.e., ensemble learning, may have a better performance. In this study, we conduct a comparative assessment of the performance of three popular ensemble methods, i.e., Bagging, Boosting, and Stacking, based on four base learners, i.e., Logistic Regression Analysis (LRA), Decision Tree (DT), Artificial Neural Network (ANN) and Support Vector Machine (SVM). Experimental results reveal that the three ensemble methods can substantially improve individual base learners. In particular, Bagging performs better than Boosting across all credit datasets. Stacking and Bagging DT in our experiments, get the best performance in terms of average accuracy, type I error and type II error.