	US 12,260,492 B2
	Method and apparatus for training a three-dimensional face reconstruction model and method and apparatus for generating a three-dimensional face image
Di Wang, Beijing (CN); Ruizhi Chen, Beijing (CN); Chen Zhao, Beijing (CN); Jingtuo Liu, Beijing (CN); Errui Ding, Beijing (CN); Tian Wu, Beijing (CN); and Haifeng Wang, Beijing (CN)
Assigned to Beijing Baidu Netcom Science Technology Co., Ltd., Beijing (CN)
Filed by Beijing Baidu Netcom Science Technology Co., Ltd., Beijing (CN)
Filed on Jan. 20, 2023, as Appl. No. 18/099,602.
Claims priority of application No. 202210738050.4 (CN), filed on Jun. 28, 2022.
Prior Publication US 2023/0419592 A1, Dec. 28, 2023
Int. Cl. G06T 15/00 (2011.01); G06T 15/04 (2011.01); G06T 15/20 (2011.01); G06T 15/40 (2011.01); G06T 19/20 (2011.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06V 40/16 (2022.01)

CPC G06T 15/20 (2013.01) [G06T 15/04 (2013.01); G06T 15/40 (2013.01); G06T 19/20 (2013.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06V 40/168 (2022.01); G06T 2219/2004 (2013.01); G06T 2219/2012 (2013.01); G06T 2219/2016 (2013.01)]

16 Claims

1. A method for training a three-dimensional face reconstruction model, comprising:

acquiring a sample face image and a stylized face map of the sample face image;

inputting the sample face image into a three-dimensional face reconstruction model to obtain a coordinate transformation parameter and a face parameter of the sample face image;

determining a three-dimensional stylized face image of the sample face image according to the face parameter of the sample face image and the stylized face map of the sample face image;

transforming the three-dimensional stylized face image of the sample face image into a camera coordinate system based on the coordinate transformation parameter, and rendering the transformed three-dimensional stylized face image to obtain a rendered map; and

training the three-dimensional face reconstruction model according to the rendered map and the stylized face map of the sample face image;

wherein determining the three-dimensional stylized face image of the sample face image according to the face parameter of the sample face image and the stylized face map of the sample face image comprises:

constructing a three-dimensional face image of the sample face image based on the face parameter of the sample face image; and

processing the three-dimensional face image of the sample face image according to the stylized face map of the sample face image to obtain the three-dimensional stylized face image of the sample face image;

wherein processing the three-dimensional face image of the sample face image according to the stylized face map of the sample face image to obtain the three-dimensional stylized face image of the sample face image comprises:

performing texture expansion on the stylized face map of the sample face image to obtain an initial texture map;

performing at least one of occlusion removal processing, highlight removal processing, or face pose adjustment processing on the initial texture map based on a map regression network to obtain a target texture map, wherein the map regression network is a pre-trained convolutional neural network for processing the initial texture map; and

processing the three-dimensional face image of the sample face image according to the target texture map to obtain the three-dimensional stylized face image of the sample face image.