A regularization-based deep unsupervised model for affine multimodal co-registration

Extensive network receptive field is key for unsupervised affine registration because instead of deformable registration that takes care of local subtleties, the affine registration is global so that the last layers need to see big patches of the organ-in-interest. To extend the network's receptive field, we need to go for deeper networks, which causes producing complex models. On the other hand, affine transformation is restricted by its low degree-of-freedom (DoF) where larger models increasingly develop the hazard of overfitting. To worsen the situation, the regularizer module cannot be applied to the affine transformation with such a restricted DoF. In this paper, we propose a differentiable computational layer to convert the affine transformation outputted by the network to its corresponding dense displacement field. Such an affine-to-field layer enables us to apply different regularization terms on the outputted transformation in order to avoid the overfitting phenomenon while deepening the network. The proposed approach was evaluated on an annotated hard multimodal dataset containing 1109 pairs of CT/MR images of the brain with different heterogeneity for example, variety in scanners, setups and resolutions. Based on the results, the proposed customized layer is fully successful to handle the overfitting for deeper networks that are able to produce richer transformations than the shallower networks from different evaluation metrics for example, in target registration error the proposed network with seven layers has a 13.3% (or 9.1 mm) improvement in performance. The implementation of the proposed customized affine-to-field layer in the Python, Keras package with the Tensorflow backend can be publically accessed via https://github.com/boveiri/Deep-coReg .