特征重加权U-Net超声图像分割的骨结构重建与增强现实显示

在现有骨科微创手术中,医生通过观察屏幕上显示的二维骨结构X射线图像,引导手术器械置入并完成治疗.但这种方式不仅缺乏直观的三维信息,而且导致病人和医生暴露在大量的射线电离辐射下.为了解决该问题,提出特征重加权超声分割的骨结构重建与增强现实显示技术,将骨结构直观展示给医生,为医生提供无辐射的手术引导.首先,开发了一个特征重加权U-Net,从Free-hand超声图像中精确分割提取骨表面.然后,根据超声图像的姿态,对分割后的骨表面进行重建,获得三维骨结构.最后,使用立体全像技术对重建的骨结构进行增强现实显示,展示给医生.在仿体和真实患者超声数据集上进行了实验,所获得的骨结构分割精度为(88.51±1.44)%,骨结构重建误差为(1.29±0.11)mm.实验结果表明,所提出的基于特征重加权U-Net超声图像分割的骨结构重建与增强现实显示技术可以为外科医生提供直观的术中三维导航信息.

3D Reconstruction and Augmented Reality Display of Bone Structure Based on Ultrasound Segmentation with Feature-Reweighted U-Net

During the minimally invasive orthopedic surgery,surgeons insert surgical instruments under the guidance of two-dimensional(2D)X-ray images of bone structures that displayed on a screen.This method not only lacks of intuitive three-dimensional(3D)information,but also exposes patients and surgeons to significant amounts of ionizing radiations.To solve these problems,this paper proposes 3D reconstruction and augmented reality display of bone structure based on Free-hand ultrasound segmentation with feature-reweighted U-Net,and provides doctors with 3D bone structure for radiation-free surgical navigation.Firstly,a feature-reweighted U-Net is developed for accurate segmentation of bone surfaces from Free-hand ultrasound scans.Then,the segmented bone surfaces are reconstructed to acquire 3D bone structure according to the locations of ultrasound images.Finally,the bone structure is further rendered using integral photography techniques for augmented reality guidance.The experiments are finished on ultrasound data set of phantom and patients.The bone segmentation accuracy of(88.51±1.44)%and bone reconstruction error of(1.29±0.11)mm are acquired during the experiments.The proposed method can provide surgeons with intuitive intraoperative 3D navigation information.