基于异构低秩多模态融合网络的后囊膜混浊预测

针对后囊膜混浊并发症发病周期长、筛查范围广的问题，提出利用多模态机器学习预测后囊膜混浊并发症的计算机辅助诊断方法. 对后照影像进行感兴趣区域(ROI)提取和白色反光区域填充，所构建的异构低秩多模态融合网络(HLMF)能同时输入后照影像和视觉质量参数进行特征提取与融合，HLMF模型基于通道积融合多模态信息；采用卷积核参数低秩分解解决过拟合问题；选用Focal Loss损失函数解决类别不均衡的问题；在训练过程中还采用预训练和模态腐蚀的训练方法，使模型更好地提取单一模态的特征并进行融合. 该算法在后囊膜混浊数据集上的十折交叉验证准确率为95.63%，F1分数为96.72%. 实验结果表明，所提算法能较好地提取单模态特征并进行特征融合，相比于其他多模态融合模型有更好的性能.

Prediction of posterior capsular opacification based on heterogeneous low-rank multimodal fusion network

A computer-aided diagnosis method for posterior capsular opacification using multimodal machine learning was proposed for the long incidence cycle and wide screening range of the complication of posterior capsular opacification. The region of interest (ROI) was extracted and the white reflective region was filled on the retro-illumination image. Retro-illumination image and visual quality data can be input into constructed heterogeneous low-rank multimodal fusion network (HLMF) simultaneously for performing feature extraction and fusion, and multimodal information was fused based on channel product. Overfitting problem was solved by low-rank decomposition of convolution kernel parameters, and the Focal Loss was chosen to solve the problem of uneven category. The pre-training and corrupted augmentation methods were used in the training process to better extract and fuse the features of single modality. The accuracy and F1 score of 10-fold cross-validation of the algorithm on the posterior capsule opacification dataset were 95.63% and 96.72%, respectively. Experimental results show that the proposed algorithm can extract single modality features and perform feature fusion well, and has better performance compared with other multimodal fusion models.