US 11,941,770 B2
System and method for 3D try-on based on human pose and body shape estimation
Xiaochuan Fan, Milpitas, CA (US); Dan Miao, San Jose, CA (US); and Chumeng Lyu, Mountain View, CA (US)
Assigned to BEIJING WODONG TIANJUN INFORMATION TECHNOLOGY CO., LTD., Beijing (CN); and JD.COM AMERICAN TECHNOLOGIES CORPORATION, Mountain View, CA (US)
Filed by Beijing Wodong Tianjun Information Technology Co., Ltd., Beijing (CN); and JD.com American Technologies Corporation, Mountain View, CA (US)
Filed on Dec. 30, 2020, as Appl. No. 17/138,213.
Prior Publication US 2022/0207596 A1, Jun. 30, 2022
Int. Cl. G06T 19/20 (2011.01); G06N 3/045 (2023.01); G06N 3/08 (2023.01); G06N 20/00 (2019.01); G06T 7/00 (2017.01); G06T 7/55 (2017.01); G06T 15/20 (2011.01); G06T 17/00 (2006.01); G06T 19/00 (2011.01)
CPC G06T 19/20 (2013.01) [G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06N 20/00 (2019.01); G06T 7/55 (2017.01); G06T 7/97 (2017.01); G06T 15/205 (2013.01); G06T 17/00 (2013.01); G06T 19/00 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
capturing, by a camera, a three dimensional (3D) image of a customer;
obtaining, by a computing device, a first 3D pose of the customer in the 3D image;
generating, by the computing device performing a machine learning model on the first 3D pose, a first skinned multi-person linear model (SMPL) pose, wherein the first SMPL pose comprises a plurality of rotation matrixes representing rotations of the customer's joints and a global rotation matrix representing whole body rotation of the customer;
calculating, by the computing device, an angle of the whole body rotation of the customer based on the global rotation matrix;
when the angle of the whole body rotation is in a predefined range relative to a front direction of the customer: constructing an SMPL model using the first SMPL pose; and
when the angle of the whole body rotation is out of the predefined range: generating a second SMPL pose using two dimensional (2D) component of the 3D image, and constructing the SMPL model using the second SMPL pose.