US 12,112,426 B2
Augmenting a view of a real-world environment with a view of a volumetric video object
Cornelius Hellge, Berlin (DE); Thomas Schierl, Berlin (DE); Peter Eisert, Berlin (DE); Anna Hilsmann, Berlin (DE); Robert Skupin, Berlin (DE); Yago Sanchez, Berlin (DE); Jangwoo Son, Berlin (DE); Gurdeep Singh Bhullar, Berlin (DE); Serhan Gül, Berlin (DE); and Dimitri Podborski, Berlin (DE)
Assigned to Fraunhofer-Gessellschaft zur Förderung der angewandten Forschung EV, Munich (DE)
Appl. No. 17/784,883
Filed by FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V., Munich (DE)
PCT Filed Dec. 15, 2020, PCT No. PCT/EP2020/086121
§ 371(c)(1), (2) Date Jun. 13, 2022,
PCT Pub. No. WO2021/122529, PCT Pub. Date Jun. 24, 2021.
Claims priority of application No. 19216574 (EP), filed on Dec. 16, 2019.
Prior Publication US 2023/0008125 A1, Jan. 12, 2023
Int. Cl. G06T 15/20 (2011.01); G06T 7/194 (2017.01); G06T 7/70 (2017.01); G06T 15/04 (2011.01); G06T 15/08 (2011.01); G06T 15/60 (2006.01); G06T 19/00 (2011.01)
CPC G06T 15/205 (2013.01) [G06T 7/194 (2017.01); G06T 7/70 (2017.01); G06T 15/04 (2013.01); G06T 15/08 (2013.01); G06T 15/60 (2013.01); G06T 19/006 (2013.01); G06T 2207/20112 (2013.01); G06T 2215/16 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method of augmenting a view of a real-world environment with a view of a volumetric video object on a user device, the method comprising the steps of:
determining a current pose information (CPI) indicating a current pose of the view of the real-world environment and a desired pose of the volumetric video object in the real-world environment;
sending the current pose information to a remote server;
receiving a rendered view of the volumetric video object that has been rendered in accordance with the current pose information from the remote server; and
augmenting the view of the real-world environment by at least mapping the rendered view of the volumetric video object onto a planar mapping surface arranged according to the desired position of the volumetric video object;
wherein the planar mapping surface is oriented such that the rendered view of the volumetric video object is projected into the view of the real-world environment substantially free of perspective distortion.