相关对齐的总变分风格迁移新模型

目的 图像的风格迁移是近年来机器视觉领域的研究热点之一。针对传统基于卷积神经网络（CNN）的图像风格迁移方法得到的结果图像存在风格纹理不均匀、噪声增强及迭代时间长等问题，本文在CNN框架下提出了一种基于相关对齐的总变分图像风格迁移新模型。方法 在详细地分析了传统风格迁移方法的基础上，新模型引入了基于相关对齐的风格纹理提取方法，通过最小化损失函数，使得风格信息更加均匀地分布在结果图像中。通过分析比较CNN分解图像后不同卷积层的重构结果，提出了新的卷积层选择策略，以有效地提高风格迁移模型的效率。新模型引入了经典的总变分正则，以有效地抑制风格迁移过程中产生的噪声，使结果图像具有更好的视觉效果。结果 仿真实验结果说明，相对于传统方法，本文方法得到的结果图像在风格纹理和内容信息上均有更好的表现，即在风格纹理更加均匀细腻的基础上较好地保留了内容图像的信息。另外，新模型可以有效地抑制风格迁移过程中产生的噪声，且具有更高的运行效率（新模型比传统模型迭代时间减少了约30%）。结论 与传统方法相比，本文方法得到的结果图像在视觉效果方面有更好的表现，且其效率明显优于传统的风格迁移模型。

Correlation alignment total variation model and algorithm for style transfer

Objective The style transfer of images has been a research hotspot in computer vision and image processing in recent years. The image style transfer technology can transfer the style of the style image to the content image, and the obtained result image contains the main content structure information of the content image and the style information of the style image, thereby satisfying people''s artistic requirements for the image. The development of image style transfer can be divided into two phases. In the first phase, people often use non-photorealistic rendering methods to add artistic style to the design works. These methods only use the low-level features of the image for style transfer, and most of them have problems, such as poor visual effects and low operational efficiency. In the second phase, researchers have performed considerable meaningful work by introducing the achievements of deep learning to style transfer. In the framework of convolutional neural networks, Researchers proposed a classical image style transfer method, which uses convolutional neural networks to extract advanced features of style and content images, and obtained the stylized result image by minimizing the loss function. Compared with the traditional non-photorealistic rendering method, the convolutional neural network-based method does not require user intervention in the style transfer process, is applicable to any type of style image, and has good universality. However, the resulting image has uneven texture expression and increased noise, and the method is more complex than other traditional methods. To address these problems, we propose a new model of total variational style transfer based on correlation alignment from a detailed analysis of the traditional style transfer method. Method In this study, we design a style texture extraction method based on correlation alignment to make the style information evenly distributed on the resulting image. In addition, the total variational regularity is introduced to suppress the noise generated during the style transfer effectively, and a more efficient result image convolution layer selection strategy is adopted to improve the overall efficiency of the new model. We build a new model consisting of three VGG-19 networks. Only the cov4_3 convolutional layer of the VGG(visual geometry group)-style network is used to provide style information. Only the cov4_2 convolutional layer of the VGG content network is used to provide content information. For a given content image c and style image s, suppose the resulting image of the style transfer is x (using a content image containing random noise as an initial value). Content image c and style image s are input into the VGG content network on the left side and the VGG style network on the right side of the new model, and the feature maps corresponding to each convolution can be obtained. The initial value of the resulting image x is input to the intermediate VGG result network, and the initial value of the feature map corresponding to each convolution layer is obtained. The Adam algorithm is used to minimize the total loss function, and the optimal value of the loss function is obtained by iteratively updating the weight of the VGG result network. The proposed style transfer model consists of three parameters, namely, content loss adjustment, style loss adjustment, and total variation regular parameters, which are set to 1, 5, and 500, respectively. All programs are coded using Python and TensorFlow deep learning framework, and experiments are performed on Alibaba Cloud GN5 cloud server. The CPU is Intel Xeon E5-2682 V4 (Broadwell) processor clocked at 2.5 GHz and has Nvidia P100 GPU with 12 GB video memory. The proposed and traditional models use the same parameters, that is, the weight ratio of content and style losses is 1:5, and the number of iterations is 5 000. Result We compare our model with the classic style transfer. Experiments show that the resulting image of the proposed model has a style texture that is close to the style image, and its content structure is close to the content image. Furthermore, the resulting image from the new model contains considerably fewer impurities than that from the Gatys model. The iteration time of new model is approximately 31 s shorter and the running efficiency is approximately 30% higher than those of the classic Gatys model. The efficiency of the proposed model is substantially improved compared with the traditional style transfer model. Moreover, a series of comparative experiments is conducted to illustrate the universality of the proposed model. Conclusion In this paper, a new model of total variational style transfer based on correlation alignment is proposed. This model introduces the method of extracting style texture based on correlation alignment and the classical total variational regularization. Thus, the style information is distributed further uniformly in the resulting image, and the noise generated in the style transfer process is effectively reduced. A new convolutional layer selection strategy is proposed by analyzing and comparing the reconstruction results of different convolutional layers after CNN decomposition images, which improves the efficiency of the style transfer model. Several experimental results show that the proposed model is superior to the classical style transfer model in terms of the visual effect of the resulting image and the operational efficiency of the algorithm.