基于跨模态空间匹配的多模态肺部肿块分割网络

现有多模态分割方法通常先对图像进行配准,再对配准后的图像进行分割.对于成像特点差异较大的不同模态,两阶段的结构匹配与分割算法下的分割精度较低.针对该问题,该文提出一种基于跨模态空间匹配的多模态肺部肿块分割网络(MMSASegNet),其具有模型复杂度低和分割精度高的特点.该模型采用双路残差U型分割网络作为骨干分割网络,...

Multi-Modal Pulmonary Mass Segmentation Network Based on Cross-Modal Spatial Alignment

Most of the existing multi-modal segmentation methods are adopted on the co-registered multi-modal images. However, these two-stage algorithms of the segmentation and the registration achieve low segmentation performance on the modalities with remarkable spatial misalignment. To solve this problem, a cross-modal Spatial Alignment based Multi-Modal pulmonary mass Segmentation Network (MMSASegNet) with low model complexity and high segmentation accuracy is proposed. Dual-path Res-UNet is adopted as the backbone segmentation architecture of the proposed network for the better multi-modal feature extraction. Spatial Transformer Networks (STN) is applied to the segmentation masks from two paths to align the spatial information of mass region. In order to realize the multi-modal feature fusion based on the spatial alignment on the region of mass, the deformed mask and the reference mask are matrix-multiplied by the feature maps of each modality respectively. Further, the yielding cross-modality spatially aligned feature maps from multiple modalities are fused and learned through the feature fusion module for the multi-modal mass segmentation. In order to improve the performance of the end-to-end multi-modal segmentation network, deep supervision learning strategy is employed with the joint cost function constraining mass segmentation, mass spatial alignment and feature fusion. Moreover, the multi-stage training strategy is adopted to improve the training efficiency of each module. On the pulmonary mass datasets containing T2-Weighted-MRI（T2W） and Diffusion-Weighted-MRI Images（DWI）, the proposed method achieved improvement on the metrics of Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD).