引入注意力机制和多视角融合的脑肿瘤MR图像U-Net分割模型

目的 脑肿瘤核磁共振（magnetic resonance，MR）图像分割对评估病情和治疗患者具有重要意义。虽然深度卷积网络在医学图像分割中取得了良好表现，但由于脑胶质瘤的恶性程度与外观表现有巨大差异，脑肿瘤MR图像分割仍是一项巨大挑战。图像语义分割的精度取决于图像特征的提取和处理效果。传统的U-Net网络以一种低效的拼接方式集成高层次特征和低层次特征，从而导致图像有效信息丢失，此外还存在未能充分利用上下文信息和空间信息的问题。对此，本文提出一种基于注意力机制和多视角融合U-Net算法，实现脑肿瘤MR图像的分割。方法 在U-Net的解码和编码模块之间用多尺度特征融合模块代替传统的卷积层，进行多尺度特征映射的提取与融合；在解码模块的级联结构中添加注意力机制，增加有效信息的权重，避免信息冗余；通过融合多个视角训练的模型引入3维图像的空间信息。结果 提出的模型在BraTS18（Multimodal Brain Tumor Segmentation Challenge 2018）提供的脑肿瘤MR图像数据集上进行验证，在肿瘤整体区域、肿瘤核心区域和肿瘤增强区域的Dice score分别为0.907、0.838和0.819，与其他方法进行对比，较次优方法分别提升了0.9%、1.3%和0.6%。结论 本文方法改进了传统U-Net网络提取和利用图像语义特征不足的问题，并引入了3维MR图像的空间信息，使得肿瘤分割结果更加准确，具有良好的研究和应用价值。

U-Net segmentation model of brain tumor MR image based on attention mechanism and multi-view fusion

Objective Magnetic resonance (MR) image segmentation of brain tumors is crucial for patient evaluation and treatment. In recent years, as the feature extraction capability of convolutional neural networks improved, deep learning technology has been applied to medical image segmentation and achieved better results than those of traditional segmentation methods. The accuracy of image semantic segmentation depends on the effect of semantic feature extraction and processing. Traditional U-Net integrates high-level and low-level features inefficiently, leading to a loss of effective image information. In addition, U-Net does not make full use of context information. This study proposes a segmentation method for brain tumor MR images based on the attention mechanism and multi-view fusion U-Net algorithm. Method The model is modified as follows. First, in order to improve the network structure, residual structure is added to the U-Net to enhance the compensation of the network low-level information to the high-level information. On the one hand, the application of the residual idea, removes the same main part of the output of each convolution layer, highlights the small changes, makes each layer of the network more sensitive to the changes of the output, and makes the training easier, on the other hand, it alleviates the problem of gradient disappearance. Second, attention mechanism is added to the cascade structure, and the weight of tumor region is increased adaptively to enhance the utilization of effective information and improve the segmentation accuracy. Third, multi-scale feature fusion module is used to replace the traditional convolution layer between the down sampling structure and up sampling structure, and hole convolution with different sampling rates is used to extract and fuse the multi-scale feature information of the image in parallel. The above is the adjustment of the network structure, and the influence of loss function and 3D structure of Brain tumor MR data set on model training is also considered in the experiment. There is a class imbalance problem in brain tumor segmentation, tumor area is smaller than normal brain tissue, network training is easily guided by a large number of irrelevant pixels, and linear combination of generalized dice loss and cross entropy loss is used to solve class imbalance problem and accelerate convergence. At the same time, Considering the influence of different view slices on the segmentation performance of the model, the three view slices of 3D MR image are trained respectively, and the multi view training model is fused in the segmentation prediction to improve the segmentation performance of the model. Result The proposed model is validated using the brain tumor MR image data set provided by Multi- modal Brain Tumor Segmentation Challenge 2018 (BraTS18), which includes four kinds of MR images of 210 high-grade gliomas(HGG) patients and 75 low-grade gliomas(LGG) patients and their real segmentation labels. The Dice similarity coefficient and Hausdorff distance95 are selected as technical indicators to further evaluate the accuracy of the brain tumor segmentation results. The effectiveness of the proposed module is proved by ablation experiments. After adding all modules, the performance of the model is optimal, the Dice scores of the entire tumor area, the core region, and the enhanced region reach 0. 883, 0. 812, and 0.774. Compared with the traditional U-Net, it is improved by 3.9%, 5.1% and 3.3% respectively, especially in the core region. After the fusion of three perspective slice training, comprehensive experiments show that the Dice scores of the entire tumor area, the core region, and the enhanced region reach 0. 907, 0. 838, and 0. 819 respectively, this algorithm exhibits better performance in terms of Dice score and Hausdorff 95 distance than others. At the same time, the slices showthe slice shows the segmentation comparison with other classical models such as FCN. In the HGG sample, the proposed method has more delicate segmentation effect in the tumor boundary area. However, the segmentation results of the LGG sample is not as good as the HGG sample, this is due to the small number of LGG training samples, which is only one third of HGG samples, proposed method is still performs better than other methods. Conclusion The segmentation method of brain tumor MR images proposed in this study improves the short- comings of the traditional U-Net network in extracting and using image semantic features, and introduces attention mechanism and multi-scale feature fusion module. The weighted mixed loss function is used to solve the class imbalance problem in brain tumor segmentation, and considering the spatial characteristics of the dataset, the multi-view model is fused to improve the segmentation performance. Experimental results show that the algorithm has good performance in the segmentation of different tumor regions and has stronger robustness, thus providing a useful reference for the clinical application of brain tumor MR image segmentation.