US 12,354,211 B2
Ray tracing between AR and real objects
Ron Weitzman, Tel Aviv (IL); Oleksandr Pyshchenko, Los Angeles, CA (US); Raz Perlman, Rehovot (IL); Vivian Su, Maynard, MA (US); and Almog Mizrahi, Tel Aviv (IL)
Assigned to SNAP INC., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Mar. 7, 2023, as Appl. No. 18/179,920.
Prior Publication US 2024/0303904 A1, Sep. 12, 2024
Int. Cl. G06T 15/06 (2011.01); G06T 19/00 (2011.01)
CPC G06T 15/06 (2013.01) [G06T 19/006 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
accessing, by a mobile device, a video depicting a first object;
obtaining, by the mobile device, a three-dimensional (3D) model of the first object,
in response to receiving input that activates a ray tracing process, applying, by the mobile device, the ray tracing process to the 3D model of the first object to estimate an optical effect on a portion of the first object relative to a second object that is depicted in the video;
modifying a visual property of the portion of the first object based on the optical effect relative to the second object;
detecting a first region of the second object in a first frame of the video;
storing a texture corresponding to the first region that is depicted in the first frame;
determining that the first region of the second object is no longer visible in a second frame of the video that is subsequent to the first frame; and
modifying the visual property of the portion of the first object depicted in the second frame using the texture of the first region that is no longer visible in the second frame.