US 11,861,811 B2
Neural network system with temporal feedback for denoising of rendered sequences
Carl Jacob Munkberg, Malmö (SE); Jon Niklas Theodor Hasselgren, Bunkeflostrand (SE); Anjul Patney, Kirkland, WA (US); Marco Salvi, Kirkland, WA (US); Aaron Eliot Lefohn, Kirkland, WA (US); and Donald Lee Brittain, Pasadena, CA (US)
Assigned to NVIDIA Corporation, Santa Clara, CA (US)
Filed by NVIDIA Corporation, Santa Clara, CA (US)
Filed on Sep. 8, 2022, as Appl. No. 17/930,668.
Application 17/930,668 is a continuation of application No. 16/717,090, filed on Dec. 17, 2019, granted, now 11,475,542.
Application 16/717,090 is a continuation in part of application No. 16/041,502, filed on Jul. 20, 2018.
Claims priority of provisional application 62/884,453, filed on Aug. 8, 2019.
Claims priority of provisional application 62/621,510, filed on Jan. 24, 2018.
Claims priority of provisional application 62/537,800, filed on Jul. 27, 2017.
Prior Publication US 2023/0014245 A1, Jan. 19, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 5/00 (2006.01); G06T 7/246 (2017.01); G06T 7/50 (2017.01)
CPC G06T 5/002 (2013.01) [G06T 7/248 (2017.01); G06T 7/50 (2017.01); G06T 2207/10016 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
receiving guide data for a rendered image frame in a sequence of rendered image frames, the sequence including a previous rendered image frame and the rendered image frame;
receiving external state including a reconstructed previous rendered image frame with fewer artifacts compared with the previous rendered image frame, wherein the external state is warped, using difference data corresponding to changes between the previous rendered image frame and the rendered image frame, to produce warped external state; and
processing the guide data for the rendered image frame using layers of a neural network model to produce a sample map that indicates a number of samples to be computed for each pixel in the rendered image frame, wherein the warped external state replaces hidden state generated by one or more of the layers of the neural network.