基于图像语义分割的真实毛笔笔触实时生成技术

书法在文化传承中占据重要地位,书法书写笔迹的生成也一直是计算机图形学的研究重点和难点.现存基于模型和经验的方法,由于建模难度大,大都将笔触表述为简单的几何图形并且缺少变化,难以真实还原毛笔书写的笔触和笔迹.使得现存书法笔迹生成软件仅仅用于娱乐,而难以上升到数字化书法教育层面.文中从计算机视觉的角度出发,通过4个相机获取毛笔的实时书写图像;针对Deeplabv3+语义分割算法无法有效地分割小尺寸类别的缺点进行优化,使用优化的Deeplabv3+算法提取图像中毛笔笔头等关键信息,并通过Hough变换和PnP位姿估计算法计算笔杆相对位姿;基于位姿信息矫正和融合各相机笔触图像,提出一种未知区域估计方法估计相机无法拍摄到的笔触区域.按照不同条件提取400多幅书写图像作为数据集并进行实验结果表明,优化后的Deeplabv3+算法平均交并比(mean intersection-over-union,mIOU)达到0.849,与优化前相比提升了0.117;在小尺寸类别上交并比(intersection-over-union,IOU)达到0.59,提升了0.473.在保证实时性的前提下,最终生成的笔触与传统基于模型和经验的方法相比,可以更加真实地还原书写时的笔触,并避免对毛笔进行复杂的建模,为笔迹生成研究提供一种新的思路.

Real-Time Brush Stroke Generation Based on Image Segmentation

Chinese Calligraphy plays an important role in Chinese culture.The simulation of real brush painting is an open issue in the field of computer graphics.Traditional methods represent strokes as simple geometric structures.It lacks change of shape because of the difficult modeling processes,which makes it difficult to simulate the real writing strokes of a brush.This paper proposed a computer vision-based method that took image data as input through four cameras.First,we built a Deeplabv3+-based algorithm by optimizing the poor segmentation performance in small size categories in order to extract the key information such as the brush.Then,the method obtained the relative pose of the penholder by the Hough transform and the PnP pose estimation.Next,the method corrected and fused the stroke images based on the pose information.We proposed an algorithm to estimate unknown regions that were not obtained by cameras.We built a dataset containing more than 400 images under various Calligraphy scenarios.The experiments of segmentation algorithm showed that our method has better mean intersection-over-union(mIOU)of 0.849,improved by 0.117 compared with the baselines.Especially in the small categories,the intersection-over-union(IOU)reached 0.59 and improved by 0.473.The experiments of final stroke generation showed that our method can produce much more realistic strokes in real-time without complex brush models.