US 12,462,486 B2
Adaptive mesh reprojection for low latency 6DOF rendering
Bernhard Reinert, Mountain View, CA (US); Sebastian Sylvan, Mountain View, CA (US); Hugues Hoppe, Mountain View, CA (US); Grigory Javadyan, Mountain View, CA (US); Egor Yusov, Mountain View, CA (US); David Chu, Mountain View, CA (US); and Kelvin Ritland, Mountain View, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC
Filed on Oct. 27, 2022, as Appl. No. 18/050,058.
Claims priority of provisional application 63/263,227, filed on Oct. 28, 2021.
Prior Publication US 2023/0134779 A1, May 4, 2023
Int. Cl. G06T 15/20 (2011.01); G06T 7/50 (2017.01); G06T 17/20 (2006.01); G06V 10/74 (2022.01)
CPC G06T 17/20 (2013.01) [G06T 7/50 (2017.01); G06V 10/761 (2022.01); G06T 2207/10024 (2013.01); G06T 2207/20164 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
predicting, by a computing system, a first point of view for a future rendering of a scene;
obtaining, by the computing system, a first frame of image data comprising a first plurality of pixels, each pixel of the first plurality of pixels associated with a respective color value and a respective depth value, the first frame of image data rendering the scene from a second point of view;
determining, by the computing system, a view error representative of differences between the first point of view and the second point of view;
selecting, by the computing system, and based on the view error, a subset of pixels of the first plurality of pixels;
generating, by the computing system, and using the subset of pixels of the first plurality of pixels, a three-dimensional (3D) polygon mesh, wherein the 3D polygon mesh includes a plurality of portions, wherein each portion of the plurality of portions is associated with a respective density of a respective plurality of vertices, wherein each vertex of the respective plurality of vertices is associated with a corresponding pixel of the subset of pixels of the first plurality of pixels, and wherein each vertex of the respective plurality of vertices has a depth in the 3D polygon mesh based on the respective depth value of the corresponding pixel;
generating, by the computing system, a second frame of image data via reprojection using the 3D polygon mesh, the second frame of image data comprising a second plurality of pixels, each pixel of the second plurality of pixels associated with a respective depth value, the second frame of image data having a third point of view different from the second point of view;
determining, by the computing system, that the second frame of image data includes a subset of pixels of the second plurality of pixels that correspond to a portion of the scene that was not visible in the first frame of image data; and
replacing, by the computing system, the respective depth value of each pixel of the subset of pixels of the second plurality of pixels with a different depth value that is smaller than the respective depth value.