US 11,741,652 B1
Volumetric avatar rendering
Gerard Cornelis Krol, Leiden (NL); and Petr Polyakov, Tampa, FL (US)
Assigned to Katmai Tech Inc., New York, NY (US)
Filed by Katmai Tech Inc., New York, NY (US)
Filed on Sep. 13, 2022, as Appl. No. 17/931,804.
Int. Cl. G06T 13/40 (2011.01); G06T 19/20 (2011.01)
CPC G06T 13/40 (2013.01) [G06T 19/20 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer implemented method for rendering a volumetric avatar in a three-dimensional (3D) virtual environment, comprising:
receiving configuration settings and a video stream with a background on a first device of a first user, wherein the configuration settings and the video stream are sent from a second device of a second user, wherein the video stream captures the second user and is captured using a camera integrated with the second device, and wherein the configuration settings comprise at least one of a video stream cutoff shape, an avatar opaqueness, a video stream edge smoothness, a background removal preference, and a framerate for background removal; and
from a perspective of a virtual camera in the 3D virtual environment corresponding to the first user:
rendering a front surface of an avatar containing the video stream, wherein the rendered front surface excludes the background, comprising:
(a) constructing a volumetric shape around the avatar, wherein the volumetric shape encompasses all the avatar;
(b) from within the volumetric shape around the avatar, calculating a shape of the avatar;
(c) excluding pixels outside of the shape of the avatar;
(d) using the configuration settings, comparing pixels within the shape of the avatar to a color of the background; and
(e) altering pixels with same color as the background;
rendering a back surface of the avatar containing a silhouette corresponding to the video stream, wherein the rendered back surface excludes the video stream and the background, comprising:
repeating operations (a)-(e);
replacing pixels that are a different color from the background with a solid color; and
calculating an intersection between the back surface and the front surface of the avatar;
rendering a side surface of the avatar, comprising:
applying a thickness to the avatar;
sampling at least one pixel within the thickness;
projecting a point corresponding to the at least one pixel onto the front surface of the avatar;
repeating operations (a)-(e); and
replacing pixels that are a different color from the background with the solid color, based on the intersection between the back surface and the front surface of the avatar, wherein the side surface is rendered as an uncharacteristic geometric shape.