基于多任务学习的多源数据分类研究

针对现有方法在处理多源数据时忽视数据源之间关联性的问题,提出了一种可以同时实现多分类效果的多源学习框架。该框架将不同的数据源看作多个相关的任务,将多源问题转换为经典的多任务学习问题,通过提取数据源之间的关联,来提高单个数据源的分类性能。此外,该框架利用聚类分析原理,对带标记样本实现多分类效果。实验结果表明,该框架优于只针对单个数据源学习的单任务学习框架和只针对二分类进行处理的传统的多任务学习框架。     

MTL-BERT:一种结合BERT的中文文本多任务学习模型

单任务学习常常受限于单目标函数的不足,多任务学习能有效利用任务相关性的先验性,故而受到了学界的关注.在中文自然语言处理领域,关于多任务学习的研究极为匮乏,该领域需同时考虑到中文文本特征提取和多任务的建模.本论文提出了一种多任务学习模型MTL-BERT.首先将BERT作为特征提取器以提升模型的泛化性.其次分类和回归是机器学习中的两个主要问题,针对多标签分类和回归的混合任务,提出了一种任务权重自适应框架.该框架下,任务之间的权重由联合模型参数共同训练.最后从模型最大似然角度,理论验证了该多任务学习算法的有效性.在真实中文数据集上的实验表明,MTL-BERT具有较好的计算效果.     

多任务学习的不平衡SVM+算法

处理不平衡数据分类时,传统支持向量机技术（SVM）对少数类样本识别率较低。鉴于SVM+技术能利用样本间隐藏信息的启发,提出了多任务学习的不平衡SVM+算法（MTL-IC-SVM+）。MTL-IC-SVM+基于SVM+将不平衡数据的分类表示为一个多任务的学习问题,并从纠正分类面的偏移出发,分别赋予多数类和少数类样本不同的错分惩罚因子,且设置少数类样本到分类面的距离大于多数类样本到分类面的距离。UCI数据集上的实验结果表明,MTL-IC-SVM+在不平衡数据分类问题上具有较高的分类精度。     

User demographics prediction based on mobile data

Demographics prediction is an important component of user profile modeling. The accurate prediction of users’ demographics can help promote many applications, ranging from web search, personalization to behavior targeting. In this paper, we focus on how to predict users’ demographics, including “gender”, “job type”, “marital status”, “age” and “number of family members”, based on mobile data, such as users’ usage logs, physical activities and environmental contexts. The core idea is to build a supervised learning framework, where each user is represented as a feature vector and users’ demographics are considered as prediction targets. The most important component is to construct features from raw data and then supervised learning models can be applied. We propose a feature construction framework, CFC (contextual feature construction), where each feature is defined as the conditional probability of one user activity under the given contexts. Consequently, besides employing standard supervised learning models, we propose a regularized multi-task learning framework to model different kinds of demographics predictions collectively. We also propose a cost-sensitive classification framework for regression tasks, in order to benefit from the existing dimension reduction methods. Finally, due to the limited training instances, we employ ensemble to avoid overfitting. The experimental results show that the framework achieves classification accuracies on “gender”, “job” and “marital status” as high as 96%, 83% and 86%, respectively, and achieves Root Mean Square Error (RMSE) on “age” and “number of family members” as low as 0.69 and 0.66 respectively, under the leave-one-out evaluation.     

Multi-task learning for concurrent survival prediction and semi-supervised segmentation of gliomas in brain MRI

Accurate survival prediction is essential for precision oncology in patients with glioma. However, current deep learning-based survival analysis methods highly rely on segmented tumor regions, which requires tedious manual annotation. Semi-supervised segmentation offers an efficient way to reduce the annotation burden. However, most studies consider survival prediction and semi-supervised segmentation as two separated problems. Here, we proposed a multi-task learning approach for concurrent survival prediction and semi-supervised tumor segmentation. We train a shared multi-modal Transformer encoder to extract features from multiple modalities and fuse them at different levels. The extracted features are employed to construct contrast learning loss and survival analysis loss to implement semi-supervised segmentation and survival analysis, respectively. Experiments are conducted on two datasets from two local hospitals. Our method achieves comparable or slightly better results than state-of-the-art semi-supervised segmentation methods and achieves acceptable survival analysis results. Our data suggests that the proposed multi-task architecture can enhance both segmentation and survival prediction tasks in a semi-supervised learning manner.     

动态事件时间数据的多任务Logistic生存预测方法

事件时间数据广泛存在于临床医学研究领域,包含大量复杂的随时间变化的动态风险因子变量。为了对这些动态事件时间数据进行有效分析,克服生存模型参数假设的局限性,提出了一种多任务Logistic生存学习和预测方法。将生存预测转化为一系列不同时间点的多任务二元生存分类问题,利用动态风险因子变量的全部观测值估计累积风险。通过对事件样本和删失样本的全数据学习正则化Logistic回归参数。评估风险因子与事件时间的动态关系,根据生存概率估计事件时间。在多个实际临床数据集上开展的对比实验验证了提出的多任务预测方法对于动态数据不仅具有较强的适用性,而且能够保障预测结果的准确性和可靠性。     

Boosted Bayesian network classifiers

The use of Bayesian networks for classification problems has received a significant amount of recent attention. Although computationally efficient, the standard maximum likelihood learning method tends to be suboptimal due to the mismatch between its optimization criteria (data likelihood) and the actual goal of classification (label prediction accuracy). Recent approaches to optimizing classification performance during parameter or structure learning show promise, but lack the favorable computational properties of maximum likelihood learning. In this paper we present boosted Bayesian network classifiers, a framework to combine discriminative data-weighting with generative training of intermediate models. We show that boosted Bayesian network classifiers encompass the basic generative models in isolation, but improve their classification performance when the model structure is suboptimal. We also demonstrate that structure learning is beneficial in the construction of boosted Bayesian network classifiers. On a large suite of benchmark data-sets, this approach outperforms generative graphical models such as naive Bayes and TAN in classification accuracy. Boosted Bayesian network classifiers have comparable or better performance in comparison to other discriminatively trained graphical models including ELR and BNC. Furthermore, boosted Bayesian networks require significantly less training time than the ELR and BNC algorithms.     

基于篇章结构多任务学习的神经机器翻译

篇章翻译方法借助跨句的上下文信息以提升篇章的翻译质量.篇章具有结构化的语义信息,可以形式化地表示为基本篇章单元之间的依存关系.但是目前的神经机器翻译方法很少利用篇章的结构信息.为此,提出了一种篇章翻译模型,能够在神经机器翻译的编码器-解码器框架中显式地建模基本篇章单元切分、篇章依存结构预测和篇章关系分类任务,从而得到结构信息增强的篇章单元表示.该表示分别通过门控加权和层次注意力的方式,与编码和解码的状态向量进行融合.此外,为了缓解模型在测试阶段对篇章分析器的依赖,在训练时采用多任务学习的策略,引导模型对翻译任务和篇章分析任务进行联合优化.在公开数据集上的实验结果表明,所提出的方法能够有效地建模和利用篇章单元间的依存结构信息,从而达到提升译文质量的目的.     

A feature-free and parameter-light multi-task clustering framework

The two last decades have witnessed extensive research on multi-task learning algorithms in diverse domains such as bioinformatics, text mining, natural language processing as well as image and video content analysis. However, all existing multi-task learning methods require either domain-specific knowledge to extract features or a careful setting of many input parameters. There are many disadvantages associated with prior knowledge requirements for feature extraction or parameter-laden approaches. One of the most obvious problems is that we may find a wrong or non-existent pattern because of poorly extracted features or incorrectly set parameters. In this work, we propose a feature-free and parameter-light multi-task clustering framework to overcome these disadvantages. Our proposal is motivated by the recent successes of Kolmogorov-based methods on various applications. However, such methods are only defined for single-task problems because they lack a mechanism to share knowledge between different tasks. To address this problem, we create a novel dictionary-based compression dissimilarity measure that allows us to share knowledge across different tasks effectively. Experimental results with extensive comparisons demonstrate the generality and the effectiveness of our proposal.     

Functional Trees

In the context of classification problems, algorithms that generate multivariate trees are able to explore multiple representation languages by using decision tests based on a combination of attributes. In the regression setting, model trees algorithms explore multiple representation languages but using linear models at leaf nodes. In this work we study the effects of using combinations of attributes at decision nodes, leaf nodes, or both nodes and leaves in regression and classification tree learning. In order to study the use of functional nodes at different places and for different types of modeling, we introduce a simple unifying framework for multivariate tree learning. This framework combines a univariate decision tree with a linear function by means of constructive induction. Decision trees derived from the framework are able to use decision nodes with multivariate tests, and leaf nodes that make predictions using linear functions. Multivariate decision nodes are built when growing the tree, while functional leaves are built when pruning the tree. We experimentally evaluate a univariate tree, a multivariate tree using linear combinations at inner and leaf nodes, and two simplified versions restricting linear combinations to inner nodes and leaves. The experimental evaluation shows that all functional trees variants exhibit similar performance, with advantages in different datasets. In this study there is a marginal advantage of the full model. These results lead us to study the role of functional leaves and nodes. We use the bias-variance decomposition of the error, cluster analysis, and learning curves as tools for analysis. We observe that in the datasets under study and for classification and regression, the use of multivariate decision nodes has more impact in the bias component of the error, while the use of multivariate decision leaves has more impact in the variance component.