Development of machine learning‐based real time scheduling systems: using ensemble based on wrapper feature selection approach

There are two items that significantly enhance the generalisation ability (i.e. classification accuracy) of machine learning‐based classifiers: feature selection (including parameter optimisation) and an ensemble of the classifiers. Accordingly, the objective in this study is to develop an ensemble of classifiers based on a genetic algorithm (GA) wrapper feature selection approach for real time scheduling (RTS). The proposed approach can better enhance the generalisation ability of the RTS knowledge base (i.e. classifier) in comparison with three classical machine learning‐based classifier RTS systems, including the GA‐based wrapper feature selection mechanism, in terms of the prediction accuracy of 10‐fold cross validation as measured according to all the performance criteria. The proposed ensemble classifier RTS also provides better system performance than the three machine learning‐based RTS systems, including the GA‐based wrapper feature selection mechanism and heuristic dispatching rules, under all the performance criteria, over a long period in a flexible manufacturing system (FMS) case study.     

基于人工鱼群算法-极限学习机的多手势精准识别

手势作为人机交互的重要方式,因灵活性与便捷性强,已成为控制领域的研究重点。针对上肢康复机器人手势识别技术存在的不足,结合特征组合与滑动窗口法,提出一种基于人工鱼群算法（artificial fish swarm algorithm,AFSA）优化的极限学习机（extreme learning machine,ELM）的多手势精准识别方法,以提高手势识别的准确率。首先,运用表面肌电测量系统采集人体常用的8种手势对应的表面肌电信号（surface electromyography,SEMG）,作为后续分类模型的信号源,并运用去噪技术与起止点检测技术对SEMG进行预处理;然后,选取通过主成分分析（principal components analysis,PCA）降维处理后的最优特征组合与最优滑动窗口;接着,采用AFSA搜寻ELM的最优输入权值和隐含阈值,以提高其分类准确率;最后,对AFSA优化的ELM（AFSA-ELM）分类模型、反向传播（back propagation,BP）神经网络分类模型和未优化的ELM分类模型进行比较,以验证所提出方法的精准性。实验结果表明,结合最优特征组合与最优滑动窗口设计的AFSA-ELM分类模型对多种手势的平均识别准确率高达97.4%,比BP神经网络分类模型和未优化的ELM分类模型分别高3.5%和1.6%,验证了所提出方法的识别精准性。研究结果可为手势识别提供新思路,进而为人体上肢动作的深度分析和上肢康复机器人手势识别算法的优化提供理论基础和参考。     

融合批量再标准化和YOLOv3的手势识别研究

针对复杂场景下手势分割准确性低,手势细粒度特征描述不充分和手势识别实时性差的问题,提出融合批量再标准化和YOLOv3的手势识别算法.首先,在复杂背景及不同光照条件下采集20种手势,运用数据增广策略进行样本扩充并建立标准手势库;然后通过K均值维度聚类获取训练集手势锚点框,负责对不同尺度手势进行检测;最后利用迁移学习和微调...     

Boosted Stacking Ensemble Machine Learning Method for Wafer Map Pattern Classification

Recently, machine learning-based technologies have been developed to automate the classification of wafer map defect patterns during semiconductor manufacturing. The existing approaches used in the wafer map pattern classification include directly learning the image through a convolution neural network and applying the ensemble method after extracting image features. This study aims to classify wafer map defects more effectively and derive robust algorithms even for datasets with insufficient defect patterns. First, the number of defects during the actual process may be limited. Therefore, insufficient data are generated using convolutional auto-encoder (CAE), and the expanded data are verified using the evaluation technique of structural similarity index measure (SSIM). After extracting handcrafted features, a boosted stacking ensemble model that integrates the four base-level classifiers with the extreme gradient boosting classifier as a meta-level classifier is designed and built for training the model based on the expanded data for final prediction. Since the proposed algorithm shows better performance than those of existing ensemble classifiers even for insufficient defect patterns, the results of this study will contribute to improving the product quality and yield of the actual semiconductor manufacturing process.     

Image Recognition of Citrus Diseases Based on Deep Learning

In recent years, with the development of machine learning and deep learning, it is possible to identify and even control crop diseases by using electronic devices instead of manual observation. In this paper, an image recognition method of citrus diseases based on deep learning is proposed. We built a citrus image dataset including six common citrus diseases. The deep learning network is used to train and learn these images, which can effectively identify and classify crop diseases. In the experiment, we use MobileNetV2 model as the primary network and compare it with other network models in the aspect of speed, model size, accuracy. Results show that our method reduces the prediction time consumption and model size while keeping a good classification accuracy. Finally, we discuss the significance of using MobileNetV2 to identify and classify agricultural diseases in mobile terminal, and put forward relevant suggestions.     

An Effective Classifier Model for Imbalanced Network Attack Data

Recently, machine learning algorithms have been used in the detection and classification of network attacks. The performance of the algorithms has been evaluated by using benchmark network intrusion datasets such as DARPA98, KDD’99, NSL-KDD, UNSW-NB15, and Caida DDoS. However, these datasets have two major challenges: imbalanced data and high-dimensional data. Obtaining high accuracy for all attack types in the dataset allows for high accuracy in imbalanced datasets. On the other hand, having a large number of features increases the runtime load on the algorithms. A novel model is proposed in this paper to overcome these two concerns. The number of features in the model, which has been tested at CICIDS2017, is initially optimized by using genetic algorithms. This optimum feature set has been used to classify network attacks with six well-known classifiers according to high f1-score and g-mean value in minimum time. Afterwards, a multi-layer perceptron based ensemble learning approach has been applied to improve the models’ overall performance. The experimental results show that the suggested model is acceptable for feature selection as well as classifying network attacks in an imbalanced dataset, with a high f1-score (0.91) and g-mean (0.99) value. Furthermore, it has outperformed base classifier models and voting procedures.     

基于Bagging算法构造强分类器的one class SVM导线舞动预测应用

考虑到传统物理分析方法无法解决导线舞动的预测问题,综合运用机器学习算法,对已有的舞动历史数据进行筛选和预处理,并挖掘有效信息,利用one class SVM算法解决舞动数据中负样本缺失问题,采用集成学习算法中Bagging算法建立分类器学习方法,实现了数据的随机抽样,分成不同组数据集进行相互独立的训练,避免对舞动数据过拟合,提升机器学习算法的抗噪声能力以及泛化能力,采用k折交叉验证算法进行模型的验证,并利用F1-score描述导线舞动预警模型的性能,验证了该方法在舞动预测方面的有效性。     

An Efficient Internet Traffic Classification System Using Deep Learning for IoT

Internet of Things (IoT) defines a network of devices connected to the internet and sharing a massive amount of data between each other and a central location. These IoT devices are connected to a network therefore prone to attacks. Various management tasks and network operations such as security, intrusion detection, Quality-of-Service provisioning, performance monitoring, resource provisioning, and traffic engineering require traffic classification. Due to the ineffectiveness of traditional classification schemes, such as port-based and payload-based methods, researchers proposed machine learning-based traffic classification systems based on shallow neural networks. Furthermore, machine learning-based models incline to misclassify internet traffic due to improper feature selection. In this research, an efficient multilayer deep learning based classification system is presented to overcome these challenges that can classify internet traffic. To examine the performance of the proposed technique, Moore-dataset is used for training the classifier. The proposed scheme takes the pre-processed data and extracts the flow features using a deep neural network (DNN). In particular, the maximum entropy classifier is used to classify the internet traffic. The experimental results show that the proposed hybrid deep learning algorithm is effective and achieved high accuracy for internet traffic classification, i.e., 99.23%. Furthermore, the proposed algorithm achieved the highest accuracy compared to the support vector machine (SVM) based classification technique and k-nearest neighbours (KNNs) based classification technique.     

基于改进判别字典学习的故障诊断方法

近年来,基于稀疏表示的分类技术在模式识别中取得一定的成功。该框架中,字典的学习和分类器的训练通常是两个独立的模块,降低了方法的识别精度。针对以上问题,提出了一种特征提取和模式识别相融合的改进判别字典学习模型,将重构误差项、稀疏编码判别项及分类误差项进行了整合,并用K奇异值分解算法对目标函数进行优化,实现了字典和分类器的同步学习。该方法先对原始信号进行经验模态分解,并从分解的本征模态函数中提取时、频特征,形成故障样本;然后将训练样本输入改进模型用K奇异值分解优化;最后用习得字典及分类器权重对测试样本进行识别。实验结果表明：该算法不但适用于小样本故障问题,而且鲁棒性和分类性能都明显高于其它算法。         

基于支持向量机的印品缺陷分类方法

目的研究印品图像的各类形状缺陷,建立基于支持向量机(Support vector machine,SVM)的印品形状缺陷分类模型。方法对印品进行符合人眼视觉特性的缺陷识别,并对提取缺陷进行特征分析。将特征数据导入支持向量机进行训练学习,SVM分类器对缺陷图像进行测试。结果分类器对点缺陷和面缺陷的识别率为100%,对线缺陷的分类准确率达93.94%。结论基于SVM的缺陷分类方法能较好地满足印品质量检测的需求。     

A novel key-frame selection-based sign language recognition framework for the video data

ABSTRACT

Sign language is a medium of communication for people with hearing disabilities. Static and dynamic gestures are identified in a video-based sign language recognition and translated them into humanly understandable phrases to achieve the communication objective. However, videos contain redundant Key-frames which require additional processing. Number of such Key-frames can be reduced. The selection of particular Key-frames without losing the required information is a challenging task. The Key-frame extraction algorithm is used which helps to speed-up the sign language recognition process by extracting essential key-frames. The proposed framework eliminates the computation overhead by picking up the distinct Key-frames for the recognition process. Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Histograms of Oriented Gradient (HOG) are used for unique features extraction. We used the bagged tree, boosted tree ensemble method, Fine KNN, and SVM for classification. We tested methodology on video-based datasets of Pakistani Sign Language. It achieved an overall 97.5% accuracy on 37 Urdu alphabets and 95.6% accuracy on 100 common words.     

Handling dataset dependence with model ensembles for skin lesion classification from dermoscopic and clinical images

Dataset dependence affects many real-life applications of machine learning: the performance of a model trained on a dataset is significantly worse on samples from another dataset than on new, unseen samples from the original one. This issue is particularly acute for small and somewhat specific databases in medical applications; the automated recognition of melanoma from skin lesion images is a prime example. We document dataset dependence in dermoscopic skin lesion image classification using three publicly available medium size datasets. Standard machine learning techniques aimed at improving the predictive power of a model might enhance performance slightly, but the gain is small, the dataset dependence is not reduced, and the best combination depends on model details. We demonstrate that simple differences in image statistics account for only 5% of the dataset dependence. We suggest a solution with two essential ingredients: using an ensemble of heterogeneous models, and training on a heterogeneous dataset. Our ensemble consists of 29 convolutional networks, some of which are trained on features considered important by dermatologists; the networks' output is fused by a trained committee machine. The combined International Skin Imaging Collaboration dataset is suitable for training, as it is multi-source, produced by a collaboration of a number of clinics over the world. Building on the strengths of the ensemble, it is applied to a related problem as well: recognizing melanoma based on clinical (non-dermoscopic) images. This is a harder problem as both the image quality is lower than those of the dermoscopic ones and the available public datasets are smaller and scarcer. We explored various training strategies and showed that 79% balanced accuracy can be achieved for binary classification averaged over three clinical datasets.     

Robust Attack Detection Approach for IIoT Using Ensemble Classifier

Generally, the risks associated with malicious threats are increasing for the Internet of Things (IoT) and its related applications due to dependency on the Internet and the minimal resource availability of IoT devices. Thus, anomaly-based intrusion detection models for IoT networks are vital. Distinct detection methodologies need to be developed for the Industrial Internet of Things (IIoT) network as threat detection is a significant expectation of stakeholders. Machine learning approaches are considered to be evolving techniques that learn with experience, and such approaches have resulted in superior performance in various applications, such as pattern recognition, outlier analysis, and speech recognition. Traditional techniques and tools are not adequate to secure IIoT networks due to the use of various protocols in industrial systems and restricted possibilities of upgradation. In this paper, the objective is to develop a two-phase anomaly detection model to enhance the reliability of an IIoT network. In the first phase, SVM and Naïve Bayes, are integrated using an ensemble blending technique. K-fold cross-validation is performed while training the data with different training and testing ratios to obtain optimized training and test sets. Ensemble blending uses a random forest technique to predict class labels. An Artificial Neural Network (ANN) classifier that uses the Adam optimizer to achieve better accuracy is also used for prediction. In the second phase, both the ANN and random forest results are fed to the model’s classification unit, and the highest accuracy value is considered the final result. The proposed model is tested on standard IoT attack datasets, such as WUSTL_IIOT-2018, N_BaIoT, and Bot_IoT. The highest accuracy obtained is 99%. A comparative analysis of the proposed model using state-of-the-art ensemble techniques is performed to demonstrate the superiority of the results. The results also demonstrate that the proposed model outperforms traditional techniques and thus improves the reliability of an IIoT network.     

Traffic Sign Recognition Method Integrating Multi-Layer Features and Kernel Extreme Learning Machine Classifier

Traffic sign recognition (TSR), as a critical task to automated driving and driver assistance systems, is challenging due to the color fading, motion blur, and occlusion. Traditional methods based on convolutional neural network (CNN) only use an end-layer feature as the input to TSR that requires massive data for network training. The computation-intensive network training process results in an inaccurate or delayed classification. Thereby, the current state-of-the-art methods have limited applications. This paper proposes a new TSR method integrating multi-layer feature and kernel extreme learning machine (ELM) classifier. The proposed method applies CNN to extract the multi-layer features of traffic signs, which can present sufficient details and semantically abstract information of multi-layer feature maps. The extraction of multi-scale features of traffic signs is effective against object scale variation by applying a new multi-scale pooling operation. Further, the extracted features are combined into a multi-scale multi-attribute vector, which can enhance the feature presentation ability for TSR. To efficiently handle nonlinear sampling problems in TSR, the kernel ELM classifier is adopted for efficient TSR. The kernel ELM has a more powerful function approximation capability, which can achieve an optimal and generalized solution for multiclass TSR. Experimental results demonstrate that the proposed method can improve the recognition accuracy, efficiency, and adaptivity to complex travel environments in TSR.     

A Novel Action Transformer Network for Hybrid Multimodal Sign Language Recognition

Sign language fills the communication gap for people with hearing and speaking ailments. It includes both visual modalities, manual gestures consisting of movements of hands, and non-manual gestures incorporating body movements including head, facial expressions, eyes, shoulder shrugging, etc. Previously both gestures have been detected; identifying separately may have better accuracy, but much communicational information is lost. A proper sign language mechanism is needed to detect manual and non-manual gestures to convey the appropriate detailed message to others. Our novel proposed system contributes as Sign Language Action Transformer Network (SLATN), localizing hand, body, and facial gestures in video sequences. Here we are expending a Transformer-style structural design as a “base network” to extract features from a spatiotemporal domain. The model impulsively learns to track individual persons and their action context in multiple frames. Furthermore, a “head network” emphasizes hand movement and facial expression simultaneously, which is often crucial to understanding sign language, using its attention mechanism for creating tight bounding boxes around classified gestures. The model’s work is later compared with the traditional identification methods of activity recognition. It not only works faster but achieves better accuracy as well. The model achieves overall 82.66% testing accuracy with a very considerable performance of computation with 94.13 Giga-Floating Point Operations per Second (G-FLOPS). Another contribution is a newly created dataset of Pakistan Sign Language for Manual and Non-Manual (PkSLMNM) gestures.     

Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar

This paper proposes a new approach to solve the problem of real-time vision-based hand gesture recognition with the combination of statistical and syntactic analyses. The fundamental idea is to divide the recognition problem into two levels according to the hierarchical property of hand gestures. The lower level of the approach implements the posture detection with a statistical method based on Haar-like features and the AdaBoost learning algorithm. With this method, a group of hand postures can be detected in real time with high recognition accuracy. The higher level of the approach implements the hand gesture recognition using the syntactic analysis based on a stochastic context-free grammar. The postures that are detected by the lower level are converted into a sequence of terminal strings according to the grammar. Based on the probability that is associated with each production rule, given an input string, the corresponding gesture can be identified by looking for the production rule that has the highest probability of generating the input string.      

Detection of pneumonia in chest X‐ray images by using 2D discrete wavelet feature extraction with random forest

Pneumonia is one of the most common and fatal diseases in the world. Early diagnosis and treatment are important factors in reducing mortality caused by the aforementioned disease. One of the most important and common techniques to diagnose pneumonia disease is the X‐ray images. By evaluating these images, various machine‐learning methods are used for accuracy in diagnosis. The presented study in this article utilizes machine‐learning techniques to evaluate these X‐ray images. The diagnosis of pediatric pneumonia is classified with a proposed machine learning method by using the chest X‐ray images. The proposed system firstly utilizes a two‐dimensional discrete wavelet transform to extract features from images. The features obtained from the wavelet method are labeled as normal and pneumonia and applied to the classifier for classification. Besides, Random Forest algorithm is used for the classification technique of 5856 X‐ray images. A 10‐fold cross‐validation method is used to evaluate the success of the proposed method and to ensure that the system avoided overfitting. By using various machine learning algorithms, simulation results reveal that the Random Forest method is proposed and it gives successful results. Results also show that, at the end of the training and validation process, the proposed method achieves higher success with an accuracy of 97.11%.     

半监督极限学习机用于Android手机活动识别的研究

基于对现有Android手机活动识别技术的分析,针对从不完全、不充分的移动传感器数据中推断人体活动的难题,将能根据无标签样本提高识别预测准确性和速度的半监督(SS)学习和体现模式分类回归的有效学习机制的极限学习机(ELM)相结合给出了解决Android手机平台的人体活动识别问题的半监督极限学习机(SS-ELM)方法,并进一步提出了主成分分析(PCA)和半监督极限学习机(SS-ELM)结合的PCA+SS-ELM新方法。实验结果表明,该方法对人体活动的识别正确率能达到95%,优于最近提出的混合专家半监督模型的正确率,从而验证了该新方法是可行性。     

Multi classifier-based score level fusion of multi-modal biometric recognition and its application to remote biometrics authentication

Biometric recognition has become a common and reliable way to authenticate the identity of a person. Multimodal biometrics has become an interest of areas for researches in the recent past as it provides more reliability and accuracy. In multimodal biometric recognition, score level fusion has been a very promising approach to improve the overall system's accuracy. In this paper, score level fusion is carried out using three categories of classifiers like, rule classifier (fuzzy classifier), lazy classifier (Naïve Bayes) and learning classifiers (ABC-NN). These three classifiers have their own advantages and disadvantages so the hybridization of classifiers leads to provide overall improvements. The proposed technique consists of three modules, namely processing module, classifier module and combination module. Finally, the proposed fusion method is applied to remote biometric authentication. The implementation is carried out using MATLAB and the evaluation metrics employed are False Acceptance Rate (FAR), False Rejection Rate (FRR) and accuracy. The proposed technique is also compared with other techniques and by employing various combinations of modalities. From the results, we can observe that the proposed technique has achieved better accuracy value and Receiver Operating Characteristic (ROC) curves when compared to other techniques. The proposed technique reached maximum accuracy of having 95% and shows the effectiveness of the proposed technique.     

基于自适应粒子群算法和支持向量机的控制图模式识别

针对目前多品种、复杂化的生产趋势,提出了一种基于自适应变异的粒子群算法（AMPSO）和支持向量机（SVM）的控制图失效模式识别的方法。利用SVM小样本学习能力,设计一对一的SVM多分类器进行控制图模式识别,并利用AMPSO算法优化SVM核函数的参数。通过对10种控制图模式(6种基本模式和4种混合模式)的20维特征仿真数据对该方法进行检验,并通过与BP、SVM、PSO SVM识别方法的对比分析。仿真试验表明该方法有效提高了控制图模式的识别精度,达到9814%,而BP仅有75%,为控制图在线实时识别提供了一种可行的途径。 