CTH-Net:从线稿和颜色点生成服装图像的
CNN-Transformer 混合网络 |
| |
作者姓名: | 潘东辉 金映含 孙旭 刘玉生 张东亮 |
| |
作者单位: | 浙江大学计算机科学与技术学院,浙江 杭州 310000 |
| |
基金项目: | 国家重点研发计划(2022YFB3303100),国家自然科学基金项目(61972340,61732015) |
| |
摘 要: | 绘制服装效果图是服装设计过程中重要的一环,针对目前存在智能化程度不足、对用户绘画水
平和想象能力要求较高等问题,提出了一种使用线稿和颜色点生成服装图像的 CNN-Transformer 混合网络
CTH-Net。CTH-Net 结合卷积神经网络(CNN)在提取局部信息和 Transformer 在处理长距离依赖方面的优势,将
2 个模型架构进行高效混合,并设计 ToPatch 和 ToFeatureMap 模块减小输入 Transformer 的数据量和维度以降
低计算资源消耗。CTH-Net 由 3 个阶段组成:一是草图阶段,旨在预测服装的颜色分布,获得没有渐变和阴影
的水彩式图像;二是细化阶段,将水彩式图像细化为有光影效果的服装图像;三是调优阶段,组合一、二阶段
的输出进一步优化生成质量。实验结果表明,仅需输入线稿和少量颜色点,CTH-Net 便能生成出高质量的服装
图像。与现有的方法相比,该网络生成图像的真实感和准确性均有较大优势。
|
CTH-Net: CNN-Transformer hybrid network for garment image
generation from sketches and color points |
| |
Authors: | PAN Dong-hui JIN Ying-han SUN Xu LIU Yu-sheng ZHANG Dong-liang |
| |
Affiliation: | College of Computer Science and Technology, Zhejiang University, Hangzhou Zhejiang 310000, China |
| |
Abstract: | Drawing garment images is an important part of garment design. To address the problems such as low
intelligence and high requirements for users? drawing skills and imagination, a CNN-Transformer hybrid network
(CTH-Net) was proposed to generate garment images from sketches and color points. CTH-Net combined the
advantages of convolutional neural networks (CNN) in extracting local information and Transformer in processing
long-range dependencies, efficiently fusing the architectures of these two models. The ToPatch and ToFeatureMap
modules were also designed to reduce the amount and dimension of data input into Transformer, thus reducing the
consumption of computing resources. CTH-Net consisted of three phases: the first drafting phase, which aimed to
predict the color distribution of garments and obtain watercolor images without gradients and shadows; the second
refinement phase, which refined the watercolor image into a realistic garment image; the third tuning phase, which
combined the outputs of the above two phases to further optimize the generation quality. The experimental results show that CTH-Net could generate high-quality garment images by simply inputting sketches and some color points.
The proposed network could outperform the existing methods in the realism and accuracy of the generated images. |
| |
Keywords: | |
|
| 点击此处可从《》浏览原始摘要信息 |
|
点击此处可从《》下载全文 |