| CPC G01S 17/894 (2020.01) [G06T 7/246 (2017.01); G06T 7/586 (2017.01); G06T 15/00 (2013.01); G06T 17/00 (2013.01); G06T 19/006 (2013.01); H04N 13/20 (2018.05); G06T 2207/10016 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/30244 (2013.01)] | 20 Claims |
|
1. A method at a 3D environment modeling system comprising:
at least one of accessing or generating a coarse 3D model of an environment, the coarse 3D model being comprised of a plurality of images of the environment stored in a memory device;
receiving a first image of the environment from a capture device;
based on determining that the first image is higher quality than a second image of the plurality of images stored in the memory device, replacing the stored second image in the memory device with the first image;
calculating a representation of surface properties in the environment from the first image, the calculation being based in part on at least one of the following: a surface normal of an element in the environment within the first image, a camera pose associated with the first image, or lighting conditions of the first image;
refining the coarse 3D model based at least in part on the representation of the surface properties in the environment within the first image; and
using the refined coarse 3D model, enabling a virtual object or a portion of the refined coarse 3D model to be projected into real space; and
using information about material properties of the refined coarse 3D model to apply inter-shadowing and reflectivity to the following: the virtual object, and to the portion of the refined coarse 3D model projected into real space.
|