US 11,989,827 B2
Method, apparatus and system for generating a three-dimensional model of a scene
Xianyu Cheng, Beijing (CN)
Assigned to Realsee (Beijing) Technology Co., Ltd., Beijing (CN)
Filed by Realsee (Beijing) Technology Co., Ltd., Beijing (CN)
Filed on Jun. 29, 2021, as Appl. No. 17/362,438.
Claims priority of application No. 202011180650.0 (CN), filed on Oct. 29, 2020.
Prior Publication US 2022/0139030 A1, May 5, 2022
Int. Cl. G06T 17/05 (2011.01); G06T 7/50 (2017.01); G06T 7/73 (2017.01); G06T 19/00 (2011.01)
CPC G06T 17/05 (2013.01) [G06T 7/50 (2017.01); G06T 7/75 (2017.01); G06T 19/006 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method for generating a 3D model of a scene, comprising:
obtaining, by a 3D modeling system comprising an imaging device, a first set of depth data, wherein the first set of data comprises pixels of a plurality of frames of depth images;
obtaining, by the 3D modeling system comprising a scanning device, a second set of depth data, wherein the second set of depth data comprises depth data points of a plurality of image frames;
determining a depth value threshold indicating a depth value;
removing one or more data points in the first set of depth data in response to the depth value of the one or more data points being greater than the depth value indicated by the depth value threshold;
determining a plurality of data pairs, wherein each of the plurality of data pairs comprises first data indicating a depth image within the first set of depth data and corresponding second data indicating an image frame within the second set of depth data, and the first data and the corresponding second data indicate an identical target object;
determining a positional relationship between the first set of depth data and the second set of depth data based on the plurality of data pairs;
receiving, by the 3D modeling system, a set of color data of the scene, wherein the set of color data comprises pixels of a plurality of color images;
generating, by the 3D modeling system, the 3D model of the scene based on the set of color data, the first set of depth data, and the second set of depth data;
causing display of the 3D model of the scene;
determining one or more image areas in the generated 3D model of the scene below a resolution threshold;
obtaining, by the imaging device comprised in the 3D modeling system, a first set of filling data comprising pixels of a plurality of depth images associated with the one or more image areas, wherein the first set of filling data is different from the first set of depth data; and
supplementing the first set of filling data to the generated 3D model to generate a new 3D model of the scene.