Forward stage‐wise ensemble regression algorithm to improve base regressors prediction ability: an empirical study

Analysis of scientific data requires accurate regressor algorithms to decrease prediction errors. Lots of machine learning algorithms, that is, neural networks, rule‐based algorithms, regression trees and some kinds of lazy learners, are used to realize this need. In recent years, different ensemble regression strategies were improved to obtain enhanced predictors with lower forecasting errors. Ensemble algorithms combine good models that make errors in different parts of analyzed data. There are mainly two approaches in ensemble regression algorithm generation; boosting and bagging. The aim of this article is to evaluate a boosting‐based ensemble approach, forward stage‐wise additive modelling (FSAM), to improve some widely used base regressors’ prediction ability. We used 10 regression algorithms in four different types to make predictions on 10 diverse data from different scientific areas and we compared the experimental results in terms of correlation coefficient, mean absolute error, and root mean squared error metrics. Furthermore, we made use of scatter plots to demonstrate the effect of ensemble modelling on the prediction accuracies of evaluated algorithms. We empirically obtained that in general FSAM enhances the accuracies of base regressors or it at least maintains the base regressor performance.     

基于双模型交互学习的半监督医学图像分割

在医学图像中,器官或病变区域的精准分割对疾病诊断等临床应用有着至关重要的作用,然而分割模型的训练依赖于大量标注数据.为减少对标注数据的需求,本文主要研究针对医学图像分割的半监督学习任务.现有半监督学习方法广泛采用平均教师模型,其缺点在于,基于指数移动平均(Exponential moving average, EMA)的参数更新方式使得老师模型累积学生模型的错误知识.为避免上述问题,提出一种双模型交互学习方法,引入像素稳定性判断机制,利用一个模型中预测结果更稳定的像素监督另一个模型的学习,从而缓解了单个模型的错误经验的累积和传播.提出的方法在心脏结构分割、肝脏肿瘤分割和脑肿瘤分割三个数据集中取得优于前沿半监督方法的结果.在仅采用30%的标注比例时,该方法在三个数据集上的戴斯相似指标(Dice similarity coefficient, DSC)分别达到89.13%, 94.15%, 87.02%.     

Improved universal quantitative steganalysis in spatial domain using ELM ensemble

Quantitative steganalysis seeks to extract the additional information about the hidden message in the covert communications. Most of the quantitative steganalyzers available in the literature target a specific embedding algorithm and generally extract the payload information using structural paradigm. Modern steganalyzers use supervised machine learning to estimate the stego payload using sophisticated feature sets. In this paper, an Ensemble Framework based universal quantitative steganalyzer for digital images is proposed which employs optimised Extreme Learning Machines as the base regressors. The framework exploits inherent diversity of the base regressor and the use of random subspaces of the image features further reduces the prediction error. The proposed ensemble regressor exhibits improved payload predictions when evaluated vis-à-vis the individual base regressor and other state-of-the-art algorithms. The experimental results across different embedding algorithms, image datasets and variedly sized feature sets demonstrate the robustness and wide applicability of the proposed framework.

     

基于样条的林火图像多阈值分割算法

基于光滑样条原理,提出一种自适应的多阈值分割算法HistSplineReg,即采用光滑样条回归图像一维直方图,再从回归函数寻找极值从而实现图像的多阈值自动分割。较之现有的阈值分割方法,HistSplineReg具有以下优势：1）设计方法符合人类直觉;2）基于光滑样条设计算法,有理论依据;3）阈值个数及阈值位置可自动选择;4）回归函数可分析求解,计算规模主要集中在矩阵的Cholesky分解,矩阵大小由图像像素水平级决定,而不是图像尺寸;5）只有一个待定参数,该参数用于平衡回归经验误差和回归函数的光滑性。对林火识别问题,实验提供一个经验参数供参考。最后,在红绿蓝颜色（RGB）模式的林火数字图像上进行实验,从灰度图像、多种颜色通道、各通道分割结果合成的彩色图像等方面进行验证,与同样采样回归思想的支持向量回归（SVR）及多项式回归（PolyFit）相比,HistSplineReg方法直观分割效果更好,且三种方法都反映出红色通道信息对林火图像分割效果的影响更为显著。     

基于编码器-解码器的半监督图像语义分割

基于深度卷积神经网络的图像语义分割方法需要大量像素级标注的训练数据,但标注的过程费时又费力.本文基于生成对抗网络提出一种编码-解码结构的半监督图像语义分割方法,其中编码器-解码器模块作为生成器,整个网络通过耦合标准多分类交叉熵损失和对抗损失进行训练.为充分利用浅层网络包含的丰富的语义信息,本文将编码器中不同尺度的特征输入到分类器,并将得到的不同粒度的分类结果融合,进而优化目标边界.此外,鉴别器通过发现无标签数据分割结果中的可信区域,以此提供额外的监督信号,来实现半监督学习.在PASCAL VOC 2012和Cityscapes上的实验表明,本文提出的方法优于现有的半监督图像语义分割方法.     

集成学习在装备小样本试验缺失数据插补上的应用

针对装备试验数据量有限和装备测试数据易缺失的现状,提出了一种基于集成学习的回归插补方法。以随机森林和XGBoost算法为回归器,通过设定快速填充基准和特征重要性评估策略的方法,改进数据子集重建和训练集与测试集的迭代划分策略,使用Optuna框架实现回归器超参数的自动优化,在某型导弹发射试验上进行实例验证。结果表明,使用集成学习算法的回归插补效果明显优于传统的统计量插补法以及KNN和BP神经网络,在不同缺失比例下的回归确定系数结果均保持在0.95以上,能有效解决装备小样本试验数据缺失的问题,并利用KEEL公测数据集验证了该方法的推广价值和通用性。     

基于半监督回归的选择性集成算法

为了提高小数据量的有标记样本问题中学习器的性能,结合半监督学习和选择性集成学习,提出了基于半监督回归的选择性集成算法SSRES。算法基于半监督学习的基本思想,同时使用有标记样本和未标记样本训练学习器从而减少对有标记样本的需求,使用选择性集成算法GRES对不同学习器进行适当的选择,并将选择的结果结合提高学习器的泛化能力。实验结果表明,在小数据量的有标记样本问题中,该算法能够有效地提高学习器的性能。     

Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

Identifying the optimal subset of regressors in a regression bagging ensemble is a difficult task that has exponential cost in the size of the ensemble. In this article we analyze two approximate techniques especially devised to address this problem. The first strategy constructs a relaxed version of the problem that can be solved using semidefinite programming. The second one is based on modifying the order of aggregation of the regressors. Ordered aggregation is a simple forward selection algorithm that incorporates at each step the regressor that reduces the training error of the current subensemble the most. Both techniques can be used to identify subensembles that are close to the optimal ones, which can be obtained by exhaustive search at a larger computational cost. Experiments in a wide variety of synthetic and real-world regression problems show that pruned ensembles composed of only 20% of the initial regressors often have better generalization performance than the original bagging ensembles. These improvements are due to a reduction in the bias and the covariance components of the generalization error. Subensembles obtained using either SDP or ordered aggregation generally outperform subensembles obtained by other ensemble pruning methods and ensembles generated by the Adaboost.R2 algorithm, negative correlation learning or regularized linear stacked generalization. Ordered aggregation has a slightly better overall performance than SDP in the problems investigated. However, the difference is not statistically significant. Ordered aggregation has the further advantage that it produces a nested sequence of near-optimal subensembles of increasing size with no additional computational cost.     

Jointly Optimized Regressors for Image Super‐resolution

Learning regressors from low‐resolution patches to high‐resolution patches has shown promising results for image super‐resolution. We observe that some regressors are better at dealing with certain cases, and others with different cases. In this paper, we jointly learn a collection of regressors, which collectively yield the smallest super‐resolving error for all training data. After training, each training sample is associated with a label to indicate its ‘best’ regressor, the one yielding the smallest error. During testing, our method bases on the concept of ‘adaptive selection’ to select the most appropriate regressor for each input patch. We assume that similar patches can be super‐resolved by the same regressor and use a fast, approximate kNN approach to transfer the labels of training patches to test patches. The method is conceptually simple and computationally efficient, yet very effective. Experiments on four datasets show that our method outperforms competing methods.     

Multi-task learning for concurrent survival prediction and semi-supervised segmentation of gliomas in brain MRI

Accurate survival prediction is essential for precision oncology in patients with glioma. However, current deep learning-based survival analysis methods highly rely on segmented tumor regions, which requires tedious manual annotation. Semi-supervised segmentation offers an efficient way to reduce the annotation burden. However, most studies consider survival prediction and semi-supervised segmentation as two separated problems. Here, we proposed a multi-task learning approach for concurrent survival prediction and semi-supervised tumor segmentation. We train a shared multi-modal Transformer encoder to extract features from multiple modalities and fuse them at different levels. The extracted features are employed to construct contrast learning loss and survival analysis loss to implement semi-supervised segmentation and survival analysis, respectively. Experiments are conducted on two datasets from two local hospitals. Our method achieves comparable or slightly better results than state-of-the-art semi-supervised segmentation methods and achieves acceptable survival analysis results. Our data suggests that the proposed multi-task architecture can enhance both segmentation and survival prediction tasks in a semi-supervised learning manner.     

深度学习在视频对象分割中的应用与展望

视频对象分割是指在给定的一段视频序列的各帧图像中,找出属于特定前景对象的所有像素点位置区域.随着硬件平台计算能力的提升,深度学习受到了越来越多的关注,在视频对象分割领域也取得了一定的进展.本文首先介绍了视频对象分割的主要任务,并总结了该任务所面临的挑战.其次,对开放的视频对象分割常用数据集进行了简要概述,并介绍了通用的...     

Learning graph-constrained cascade regressors for single image super-resolution

Learning cascade regression has been shown an effective strategy to further enhance the perceptual quality of resulted high-resolution (HR) images. However, previous cascade regression-based SR methods have two obvious weaknesses: (1)edge structures cannot be preserved well when applying texture features to represent low-resolution (LR) images, and (2)the local manifold structures spanned by the LR-HR feature spaces cannot be revealed by the learned local linear mappings. To alleviate the aforementioned problems, a novel example regression-based super-resolution (SR) approach called learning graph-constrained cascade regressors (LGCCR) is presented, which learns a group of multi-round residual regressors in a unique way. Specifically, we improve the edge preservation capability by synthesizing the whole HR image rather than local image patches, which facilitates to extract the edge features to represent LR images. Moreover, we utilize a graph-constrained regression model to build the local linear regressors, where each local linear regressor responds to an anchored atom in the learned over-complete dictionary. Both quantitative and qualitative quality evaluations on seven benchmark databases indicate the superiority of the proposed LGCCR-based SR approach in comparing with other state-of-the-art SR predecessors.

     

基于偏袒性半监督集成的SVM主动反馈方案

现有的SVM主动反馈算法普遍受到小样本问题和不对称分布问题的制约。针对这些问题,文中提出一种基于偏袒性半监督集成的SVM主动反馈技术。该算法在集成学习框架中使用未标记数据以增加个体分类器之间的差异性,从而获得高效的集成分类模型。同时,高效的集成分类模型更有利于寻找富有信息样本,进而也提高主动反馈的效率。此外,文中还设计一种偏袒加权策略,使得集成分类模型对正样本给予更大的关注程度,以应对正负样本间的不对称分布问题。实验结果表明,偏袒性半监督集成可有效改进SVM主动反馈的性能,且文中算法的检索精度明显优于其它同类相关反馈算法。     

Multiple-View Multiple-Learner Semi-Supervised Learning

Some recent successful semi-supervised learning methods construct more than one learner from both labeled and unlabeled data for inductive learning. This paper proposes a novel multiple-view multiple-learner (MVML) framework for semi-supervised learning, which differs from previous methods in possession of both multiple views and multiple learners. This method adopts a co-training styled learning paradigm in enlarging labeled data from a much larger set of unlabeled data. To the best of our knowledge it is the first attempt to combine the advantages of multiple-view learning and ensemble learning for semi-supervised learning. The use of multiple views is promising to promote performance compared with single-view learning because information is more effectively exploited. At the same time, as an ensemble of classifiers is learned from each view, predictions with higher accuracies can be obtained than solely adopting one classifier from the same view. Experiments on different applications involving both multiple-view and single-view data sets show encouraging results of the proposed MVML method.     

Automatic segmentation of optic disc in retinal fundus images using semi-supervised deep learning

Diseases of the eye require manual segmentation and examination of the optic disc by ophthalmologists. Though, image segmentation using deep learning techniques is achieving remarkable results, it leverages on large-scale labeled datasets. But, in the field of medical imaging, it is challenging to acquire large labeled datasets. Hence, this article proposes a novel deep learning model to automatically segment the optic disc in retinal fundus images by using the concepts of semi-supervised learning and transfer learning. Initially, a convolutional autoencoder (CAE) is trained to automatically learn features from a large number of unlabeled fundus images available from the Kaggle’s diabetic retinopathy (DR) dataset. The autoencoder (AE) learns the features from the unlabeled images by reconstructing the input images and becomes a pre-trained network (model). After this, the pre-trained autoencoder network is converted into a segmentation network. Later, using transfer learning, the segmentation network is trained with retinal fundus images along with their corresponding optic disc ground truth images from the DRISHTI GS1 and RIM-ONE datasets. The trained segmentation network is then tested on retinal fundus images from the test set of DRISHTI GS1 and RIM-ONE datasets. The experimental results show that the proposed method performs on par with the state-of-the-art methods achieving a 0.967 and 0.902 dice score coefficient on the test set of the DRISHTI GS1 and RIM-ONE datasets respectively. The proposed method also shows that transfer learning and semi-supervised learning overcomes the barrier imposed by the large labeled dataset. The proposed segmentation model can be used in automatic retinal image processing systems for diagnosing diseases of the eye.

     

正例半监督学习眉毛图像分割

针对传统交互图像分割方法需要同时标注背景和前景的问题,提出一种新的交互图像分割方法——正例半监督学习图像分割。该方法结合正例半监督学习和图半监督学习,仅需要在感兴趣的图像区域标记少量像素点,就可以完成该区域的分割。在北工大眉毛图像数据库上的实验表明本文提出的方法与图半监督学习、随机游走和Lazy Snapping相比具有更稳定的分割效果。     

基于支持向量机协同训练的半监督回归

将支持向量机与半监督学习理论相结合,提出基于支持向量机协同训练的半监督回归模型,使用两个支持向量机回归模型相互影响,协同训练。利用实验数据集进行实验,并与监督支持向量机回归模型、半监督自训练支持向量机回归模型作比较。实验结果表明,基于支持向量机协同训练的半监督回归模型在缺少标记样本的情况下,提高了回归估计的精度。     

Global and local scatter based semi-supervised dimensionality reduction with active constraints selection in ensemble subspaces

Semi-supervised dimensionality reduction has attracted an increasing amount of attention in this big-data era. Many algorithms have been developed with a small number of pairwise constraints to achieve performances comparable to those of fully supervised methods. However, one challenging problem with semi-supervised approaches is the appropriate choice of the constraint set, including the cardinality and the composition of the constraint set which, to a large extent, affects the performance of the resulting algorithm. In this work, we address the problem by incorporating ensemble subspaces and active learning into dimensionality reduction and propose a new global and local scatter based semi-supervised dimensionality reduction method with active constraints selection. Unlike traditional methods that select the supervised information in one subspace, we pick up pairwise constraints in ensemble subspaces, where a novel active learning algorithm is designed with both exploration and filtering to generate informative pairwise constraints. The automatic constraint selection approach proposed in this paper can be generalized to be used with all constraint-based semi-supervised learning algorithms. Comparative experiments are conducted on four face database and the results validate the effectiveness of the proposed method.     

Incorporating feature selection method into support vector regression for stock index forecasting

Stock index forecasting is one of the most difficult tasks that financial organizations, firms and private investors have to face. Support vector regression (SVR) has become a popular alternative in stock index forecasting tasks due to its generalization capability in obtaining a unique solution. However, the major limitation of SVR is that it cannot capture the relative importance of independent variables to the dependent variable when many potential independent variables are considered. This study incorporates feature selection method and SVR for building stock index forecasting model. The proposed model uses multivariate adaptive regression splines (MARS), an effective nonlinear and nonparametric regression methodology, to identify important forecasting variables. The obtained significant predictor variables are then served as the inputs for the SVR model. Experimental results reveal that the obtained important variables from MARS can improve the forecasting performance of the SVR models. Moreover, the MARS results provide useful information about the relationship between the selected predictor variables and stock index through the obtained basis functions, important predictor variables and the MARS prediction function. Hence, the proposed stock index forecasting model can generate good forecasting performance and exhibits the capability of identifying significant predictor variables, which provide valuable information for further investment decisions/strategies.     

基于半监督学习的眉毛图像分割方法

眉毛图像的分割,由于受到毛发、姿势及个体差异的影响,是一个非常困难的问题。提出了一种利用半监督学习技术进行彩色眉毛图像分割的方法,首先通过手工在眉毛图像上简单画上几条线标注部分眉毛点和非眉毛点,然后利用半监督学习技术完成眉毛图像分割并从中提取纯眉毛图像,最后通过实验说明该方法具有非常好的分割效果,可用于眉毛识别的前期预处理。