US 12,307,570 B2
Local live texturing of a 3D mesh
Flora Ponjou Tasse, London (GB); Pavan Kumar Kamaraju, London (GB); and Ghislain Fouodji Tasse, London (GB)
Assigned to STREEM, LLC, Portland, OR (US)
Filed by STREEM, LLC, Portland, OR (US)
Filed on Aug. 18, 2022, as Appl. No. 17/890,941.
Claims priority of provisional application 63/318,664, filed on Mar. 10, 2022.
Prior Publication US 2023/0290036 A1, Sep. 14, 2023
Int. Cl. G06T 15/04 (2011.01); G06T 7/11 (2017.01); G06T 17/20 (2006.01); G06T 19/20 (2011.01)
CPC G06T 15/04 (2013.01) [G06T 7/11 (2017.01); G06T 17/20 (2013.01); G06T 19/20 (2013.01); G06T 2219/2012 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, at a computing device, a sequence of frames of a scene captured by a capturing device, and adding keyframes that partially overlap in the sequence of frames to a queue of keyframes;
accessing, by the computing device, a 3D mesh created from the sequence of frames;
determining, by the computing device, when changes to a property of the 3D mesh meet a predetermined threshold;
determining, by the computing device, for each face in the 3D mesh, from which of the keyframes in the queue of keyframes the face is visible by projecting the face vertices into RGB images of the keyframes to test visibility, and adding the keyframes from which the face is visible to a visible keyframe list for each of the faces;
assigning to each of the faces in the 3D mesh, by the computing device, a most recently added keyframe from the visible keyframe list, and dividing the 3D mesh into mesh segments based on the assigned keyframes;
using, by the computing device, the keyframe assigned to each of the mesh segments to compute texture coordinates (x, y) for vertices in the respective mesh segment, and assigning an image in the keyframe as a texture for the respective mesh segment; and
averaging, by the computing device, for each of the vertices in the 3D mesh, colors from the visible keyframe list associated with each of the vertices into a single RGB value, and assigning the RGB value to the vertex.