Mutual information maximizing GAN inversion for real face with identity preservation

Recent generative adversarial networks (GANs) have yielded remarkable performance in face image synthesis. GAN inversion embeds an image into the latent space of a pretrained generator, enabling it to be used for real face manipulation. However, current inversion approaches for real faces suffer the dilemma of initialization collapse and identity loss. In this paper, we propose a hierarchical GAN inversion for real faces with identity preservation based on mutual information maximization. We first use a facial domain guaranteed initialization to avoid the initialization collapse. Furthermore, we prove that maximizing the mutual information between inverted faces and their identities is equivalent to minimizing the distance between identity features from inverted and original faces. Optimization for real face inversion with identity preservation is implemented on this mutual information-maximizing constraint. Extensive experimental results show that our approach outperforms state-of-the-art solutions for inverting and editing real faces, particularly in terms of face identity preservation.     

改进的CAGAN在虚拟试衣中的应用

针对CAGAN（Conditional Analogy GAN）换衣后效果模糊，在目标衣服与原始衣服长短不一致时效果一般，相对目标衣服保留过少的细节等问题做了相关研究并对CAGAN进行了改进，提出了新的虚拟试衣方式。经过改进的CAGAN生成一个粗糙的结果，由该结果得到目标衣服穿在模特身上改变形状后的mask，接下来利用mask对目标衣服进行变形，综合变形的衣服和第一步的结果便得到最终的试衣图像。实验结果表明，该方法解决了前面存在的问题，而且取得了非常好的效果。     

基于循环一致性对抗网络的室内火焰图像场景迁移

基于深度学习的视频火灾探测模型的训练依赖于大量的正负样本数据,即火灾视频和带有干扰的场景视频。由于很多室内场合禁止点火,导致该场景下的火灾视频样本不足。本文基于生成对抗网络,将其他相似场景下录制的火焰迁移到指定场景,以此增广限制性场合下的火灾视频数据。文中提出将火焰内核预先植入场景使之具备完整的内容信息,再通过添加烟雾和地面反射等风格信息,完成场景与火焰的融合。该方法克服了现有多模态图像转换方法在图像转换过程中因丢失信息而造成的背景失真问题。同时为减少数据采集工作量,采用循环一致性生成对抗网络以解除训练图像必须严格匹配的限制。实验表明,与现有多模态图像转换相比,本文方法可以保证场景中火焰形态的多样性,迁移后的场景具有较高的视觉真实性,所得结果的FID与LPIPS值最小,分别为119.6和0.134 2。     

A new VAE-GAN model to synthesize arterial spin labeling images from structural MRI

Arterial spin labeling (ASL) is a relatively new MRI technique that can measure cerebral blood flow, which is of great importance for the diagnosis of dementia diseases. Besides, this valuable imaging modality does not need exogenous tracers and has no radiation, which makes it favorable for elder patients. However, ASL data does lack in many contemporary image-based dementia diseases datasets, which include popular ADNI-1/GO/2/3 datasets. In order to supplement the valuable ASL data, a new Generative adversarial network (GAN)-based model is proposed to synthesize ASL images in this study. This new model is unique, as the popular variational auto-encoder (VAE) has been utilized as the generator of the GAN-based model. Hence, a new VAE-GAN architecture is introduced in this study. In order to demonstrate its superiority, dozens of experiments have been conducted. Experimental results demonstrate that, this new VAE-GAN model is superior to other state-of-the-art ASL image synthesis methods, and the accuracy improvement after incorporating synthesized ASL images from the new model can be as high as 42.41% in dementia diagnosis tasks.     

Retinal microaneurysms detection using adversarial pre-training with unlabeled multimodal images

The detection of retinal microaneurysms is crucial for the early detection of important diseases such as diabetic retinopathy. However, the detection of these lesions in retinography, the most widely available retinal imaging modality, remains a very challenging task. This is mainly due to the tiny size and low contrast of the microaneurysms in the images. Consequently, the automated detection of microaneurysms usually relies on extensive ad-hoc processing. In this regard, although microaneurysms can be more easily detected using fluorescein angiography, this alternative imaging modality is invasive and not adequate for regular preventive screening.In this work, we propose a novel deep learning methodology that takes advantage of unlabeled multimodal image pairs for improving the detection of microaneurysms in retinography. In particular, we propose a novel adversarial multimodal pre-training consisting in the prediction of fluorescein angiography from retinography using generative adversarial networks. This pre-training allows learning about the retina and the microaneurysms without any manually annotated data. Additionally, we also propose to approach the microaneurysms detection as a heatmap regression, which allows an efficient detection and precise localization of multiple microaneurysms. To validate and analyze the proposed methodology, we perform an exhaustive experimentation on different public datasets. Additionally, we provide relevant comparisons against different state-of-the-art approaches. The results show a satisfactory performance of the proposal, achieving an Average Precision of 64.90%, 31.36%, and 33.55% in the E-Ophtha, ROC, and DDR public datasets. Overall, the proposed approach outperforms existing deep learning alternatives while providing a more straightforward detection method that can be effectively applied to raw unprocessed retinal images.