|
|||||
|
|
| 基于关键点表示的语音驱动说话人脸视频生成 | |
| 引用本文： | 年福东,王文涛,王妍,张晶晶,胡贵恒,李腾.基于关键点表示的语音驱动说话人脸视频生成[J].模式识别与人工智能,2021,34(6):572-580. |
| 作者姓名： | 年福东 王文涛 王妍 张晶晶 胡贵恒 李腾 |
| 作者单位： | 1.安徽大学 人工智能学院 合肥 230601 2.合肥学院 先进制造工程学院 合肥 230601 3.安徽工商职业学院 信息工程学院 合肥 231131 |
| 基金项目： | 安徽省高校协同创新项目(No.GXXT-2019-007)、国家自然科学基金项目(No.61902104)、安徽省自然科学基金项目(No.2008085QF295)、安徽高校自然科学研究项目(No.KJ2020A0651)、合肥学院人才科研基金项目(No.18-19RC54) |
| 摘 要： | 针对现有语音生成说话人脸视频方法忽略说话人头部运动的问题,提出基于关键点表示的语音驱动说话人脸视频生成方法.分别利用人脸的面部轮廓关键点和唇部关键点表示说话人的头部运动信息和唇部运动信息,通过并行多分支网络将输入语音转换到人脸关键点,通过连续的唇部关键点和头部关键点序列及模板图像最终生成面部人脸视频.定量和定性实验表明,文中方法能合成清晰、自然、带有头部动作的说话人脸视频,性能指标较优. |
| 关 键 词： | 说话人脸 人脸关键点 唇部动作 头部动作 人脸视频 |
| 收稿时间： | 2021-03-03 |
Speech Driven Talking Face Video Generation via Landmarks Representation |
|
| NIAN Fudong,WANG Wentao,WANG Yan,ZHANG Jingjing,HU Guiheng,LI Teng.Speech Driven Talking Face Video Generation via Landmarks Representation[J].Pattern Recognition and Artificial Intelligence,2021,34(6):572-580. | |
| Authors: | NIAN Fudong WANG Wentao WANG Yan ZHANG Jingjing HU Guiheng LI Teng |
| Affiliation: | 1. School of Artificial Intelligence, Anhui University, Hefei 230601 2. School of Advanced Manufacturing Engineering, Hefei University, Hefei 230601 3. School of Information Engineering, Anhui Business and Technology College, Hefei 231131 |
| Abstract: | The speaker's head motion is ignored in the existing speech driven talking face video generation methods. Aiming at this problem, a speech driven talking face video generation method based on facial landmarks representation is proposed. The speaker's head motion information and lip motion information are represented by facial contour landmarks and lip landmarks, respectively. The speech is converted to facial landmarks through a parallel multi-branch network. The final talking face video is synthesized by continuous lip landmark sequence, head landmark sequence and template image. The corresponding quantitative and qualitative experiments are conducted. Experimental results show that the talking face video with head action synthesized by the proposed method is clear and natural, and its performance is better. |
| Keywords: | Talking Face Facial Landmark Lip Action Head Action Face Video |
| 点击此处可从《模式识别与人工智能》浏览原始摘要信息 | |
| 点击此处可从《模式识别与人工智能》下载全文 | |