US 12,307,732 B2
Methods for handling occlusion in augmented reality applications using memory and device tracking and related apparatus
Diego Gonzalez Morin, Galapagar (ES); Simona Gugliermo, Stockholm (SE); José Araújo, Stockholm (SE); AmirHossein Taher Kouhestani, Solna (SE); Ioannis Karagiannis, Kalamata (GR); and Ananya Muddukrishna, Enskededalen (SE)
Assigned to Telefonaktiebolaget LM Ericsson (publ), Stockholm (SE)
Appl. No. 17/784,697
Filed by Telefonaktiebolaget LM Ericsson (publ), Stockholm (SE)
PCT Filed Dec. 13, 2019, PCT No. PCT/EP2019/085191
§ 371(c)(1), (2) Date Jun. 13, 2022,
PCT Pub. No. WO2021/115623, PCT Pub. Date Jun. 17, 2021.
Prior Publication US 2023/0014448 A1, Jan. 19, 2023
Int. Cl. G06V 10/26 (2022.01); G06T 5/30 (2006.01); G06T 7/194 (2017.01); G06T 7/246 (2017.01); G06T 7/90 (2017.01); G06T 15/40 (2011.01); G06T 19/00 (2011.01); G06V 10/764 (2022.01); G06V 20/20 (2022.01)
CPC G06V 10/26 (2022.01) [G06T 5/30 (2013.01); G06T 7/194 (2017.01); G06T 7/248 (2017.01); G06T 7/90 (2017.01); G06T 15/40 (2013.01); G06T 19/006 (2013.01); G06V 10/764 (2022.01); G06V 20/20 (2022.01); G06T 2200/04 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/20076 (2013.01); G06T 2207/20221 (2013.01)] 23 Claims
OG exemplary drawing
 
1. A method for occlusion handling in augmented reality performed by a device, the method comprising:
generating at least one pixel classification image in a frame,
wherein the at least one pixel classification image includes an occluding object and has pixels comprising a foreground with pixels classified as foreground pixels, a background with pixels classified as background pixels, and unknown pixels,
wherein the generating comprises (1) calculating an initial foreground pixel probability image defining, for each of one or more of the pixels of an image containing the occluding object, a probability from a foreground color probability distribution that the pixel is a foreground pixel, and an initial background pixel probability image defining, for each of one or more of the pixels, on a probability that the each of one or more of the pixels is static or dynamic, (2) calculating a normalized depth image of the occluding object based on depth information of the occluding object, and (3) computing a weighted background pixel probability image and a weighted foreground pixel probability image based on the initial background probability image and the initial foreground pixel probability image, the normalized depth image, and a variability of the color and the depth of the one or more pixels;
initiating an alpha matting process to obtain an alpha mask to blend a virtual object and the foreground of the at least one pixel classification image based on determining a color of the unknown pixels; and
rendering a final composition of an augmented reality image containing the virtual object occluded by the occluding object based on applying the alpha mask to pixels in the at least one pixel classification image.