CPC G06T 19/006 (2013.01) [G06F 18/214 (2023.01); G06T 3/0093 (2013.01); G06T 17/20 (2013.01); G06T 19/20 (2013.01); G06V 20/176 (2022.01); G06V 20/647 (2022.01); G06T 2219/2024 (2013.01)] | 20 Claims |
1. A computer-implemented method, comprising:
receiving a two-dimensional (2D) image and metadata, the 2D image including a set of pixels depicting a physical structure captured by an image capturing device, and the metadata representing one or more characteristics of the image capturing device;
identifying a portion of the 2D image to augment with synthetic image data;
segmenting the set of pixels of the 2D image into one or more subsets of pixels;
identifying, from amongst the one or more subsets of pixels, a subset of pixels corresponding to the identified portion of the 2D image;
generating a reference three-dimensional (3D) model of the physical structure from the 2D image, the reference 3D model representing the identified portion of the 2D image in a virtual space, and the generation including determining a 3D orientation of a 3D planar surface of the reference 3D model, wherein determining the 3D orientation comprises:
associating the 3D planar surface with the identified portion of the 2D image to augment with the synthetic image data;
extracting a surface normal value of the identified portion of the 2D image; and
orienting the reference 3D model to align the 3D planar surface according to the extracted surface normal;
applying the synthetic image data onto the reference 3D model;
reprojecting a select 3D geometry of the reference 3D model with the synthetic image data over the portion of the 2D image according to the identified subset of pixels; and
rendering a photorealistic image using the 2D image, the metadata, and the reprojected selected 3D geometry with the synthetic image data.
|