US 12,340,518 B2
Figure-ground neural radiance fields for three-dimensional object category modelling
Matthew Alun Brown, Vancouver (CA); Ricardo Martin-Brualla, Seattle, WA (US); Keunhong Park, Seattle, WA (US); and Christopher Derming Xie, London (GB)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on May 21, 2024, as Appl. No. 18/669,733.
Application 18/669,733 is a continuation of application No. 17/551,724, filed on Dec. 15, 2021, granted, now 12,026,892.
Claims priority of provisional application 63/270,118, filed on Oct. 21, 2021.
Prior Publication US 2024/0303825 A1, Sep. 12, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 7/194 (2017.01); G06T 17/00 (2006.01)
CPC G06T 7/194 (2017.01) [G06T 17/00 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, the method comprising:
obtaining, by a computing system comprising one or more processors, a foreground neural radiance field model, wherein the foreground neural radiance field model was trained to render one or more objects from a particular object category based on a loss function that evaluates differences between outputs of the foreground neural radiance field model and a plurality of training images, wherein the plurality of training images are descriptive of the one or more objects in the particular object category;
obtaining, by the computing system, an input position, an input view direction, and an object embedding, wherein the object embedding is associated with a particular object in the particular object category;
processing, by the computing system, the input position, the input view direction, and the object embedding with the foreground neural radiance field model to generate a foreground output, wherein the foreground output comprises a foreground color output and a foreground density output;
processing, by the computing system, the input position and the input view direction with a background neural radiance field model to generate a background output, wherein the background output comprises a background color output and a background density output; and
generating, by the computing system, a rendering comprising the particular object in an environment based at least in part on the foreground output and the background output.