US 12,254,577 B2
Pixel depth determination for object
Madiyar Aitbayev, London (GB); Brian Fulkerson, London (GB); Riza Alp Guler, London (GB); Georgios Papandreou, London (GB); and Himmy Tam, London (GB)
Assigned to SNAP INC., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Jun. 16, 2022, as Appl. No. 17/842,006.
Claims priority of application No. 220100299 (GR), filed on Apr. 5, 2022.
Prior Publication US 2023/0316666 A1, Oct. 5, 2023
Int. Cl. G06T 19/00 (2011.01); G06N 20/20 (2019.01); G06V 10/26 (2022.01); G06V 10/70 (2022.01); G06V 20/20 (2022.01)
CPC G06T 19/006 (2013.01) [G06N 20/20 (2019.01); G06V 10/26 (2022.01); G06V 10/70 (2022.01); G06V 20/20 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, by one or more processors of a device, an image that includes data representing a depiction of a person;
extracting a portion of the image corresponding to the data representing the depiction of the person;
applying a first machine learning model stage to the portion of the image to predict a depth of a point of interest for the data representing the depiction of the person;
predicting, by a second machine learning model stage, a relative depth of each pixel in the portion of the image to the depth of the point of interest predicted by the applying of the first machine learning model stage to the portion of the image, the first and second machine learning model stages processing the same image;
generating dense depth reconstruction of the data representing the depiction of the person based on outputs of the first and second stages of the machine learning model;
determining, based on the dense depth reconstruction, a second distance between an individual pixel in the portion of the image and a camera; and
applying one or more augmented reality (AR) elements to the image based on the dense depth reconstruction and the second distance comprising comparing the second distance with a first distance between the camera and an AR element of the one or more AR elements.