US 12,260,483 B2
Learning neural reflectance shaders from images
Benjamin Ummenhofer, Unterhaching (DE); Shenlong Wang, Santa Clara, CA (US); Sanskar Agrawal, Santa Clara, CA (US); Yixing Lao, Santa Clara, CA (US); Kai Zhang, Santa Clara, CA (US); Stephan Richter, Neubiberg (DE); and Vladlen Koltun, Santa Clara, CA (US)
Assigned to Intel Corporation, Santa Clara, CA (US)
Filed by Intel Corporation, Santa Clara, CA (US)
Filed on Jan. 30, 2024, as Appl. No. 18/426,740.
Application 18/426,740 is a continuation of application No. 17/849,055, filed on Jun. 24, 2022, granted, now 11,972,519.
Claims priority of provisional application 63/334,588, filed on Apr. 25, 2022.
Prior Publication US 2024/0221277 A1, Jul. 4, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 15/00 (2011.01); G06N 3/045 (2023.01); G06T 15/04 (2011.01); G06T 15/50 (2011.01); G06T 17/20 (2006.01)
CPC G06T 15/005 (2013.01) [G06N 3/045 (2023.01); G06T 15/04 (2013.01); G06T 15/506 (2013.01); G06T 17/20 (2013.01); G06T 2200/08 (2013.01); G06T 2210/52 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An apparatus comprising:
a host interface; and
one or more processors coupled with the host interface, the one or more processors including a graphics processor, wherein the one or more processors are configured to execute instructions stored in a memory, the instructions to cause the one or more processors to:
process, via at least one machine learning model, at least one input image;
generate, via the at least one machine learning model, a set of latent codes for an object within the at least one input image, wherein the set of latent codes includes an illumination latent code to represent lighting information for the object, and reflectance latent codes to represent surface reflectance for the object; and
render a three-dimensional representation of the object via a real-time renderer by shading a mesh via sampled latent codes from the set of latent codes.