US 12,283,023 B1
Neural networks for high dynamic range video super-resolution
Shruthi Suresh Rotti, Mountain House, CA (US); and Anustup Kumar Atanu Choudhury, Campbell, CA (US)
Assigned to Dolby Laboratories Licensing Corporation, San Francisco, CA (US)
Appl. No. 18/846,430
Filed by Dolby Laboratories Licensing Corporation, San Francisco, CA (US)
PCT Filed Mar. 17, 2023, PCT No. PCT/US2023/064624
§ 371(c)(1), (2) Date Sep. 12, 2024,
PCT Pub. No. WO2023/183756, PCT Pub. Date Sep. 28, 2023.
Claims priority of provisional application 63/323,348, filed on Mar. 24, 2022.
Claims priority of application No. 22169403 (EP), filed on Apr. 22, 2022.
Int. Cl. G06T 3/40 (2024.01); G06T 3/4046 (2024.01); G06T 3/4053 (2024.01)
CPC G06T 3/4046 (2013.01) [G06T 3/4053 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A method for super-resolution of pictures in a video sequence:
receiving at a first spatial resolution a current picture (102) and auxiliary pictures (104), wherein the auxiliary pictures (104) comprise a first auxiliary picture (104-1) prior to or subsequent to the current picture (102) and a second auxiliary picture (104-2) prior to or subsequent to the current picture (102);
generating a neural network, NN, features output based on a features extraction neural network (105), the current picture (102) and the auxiliary pictures (104), wherein the features extraction network (105) comprises:
a first NN path processing the current picture (102);
a second NN path comprising a first non-local residual block (205) processing the current picture (102) and the first auxiliary picture (104-1);
a third NN path comprising a second non-local residual block (205) processing the current picture (102) and a second auxiliary picture (104-2); and
wherein the features extraction network (105) generates the NN features output by:
concatenating, by a first concatenation block, outputs from the first NN path and the second NN path,
processing, by a convolutional sub-network comprising at least one convolutional block, an output of the first concatenation block,
concatenating, by a second concatenation block, an output of the convolutional sub-network and the output of the third NN path, and
concatenating, by a third concatenation block, the output of the first concatenation block and an output of the second concatenation block, and
processing, by an output convolutional block, the output of the third concatenation block to generate the NN features output;
generating a NN upscaling output at a second spatial resolution based on the NN features output and an upscaling neural network (110);
generating a NN reconstruction output at the second spatial resolution based on the NN upscaling output and a reconstruction neural network (115);
upscaling the current picture with a first picture scaler to generate a first upscaled picture at the second spatial resolution; and
adding the first upscaled picture to the NN reconstruction output to generate a super-resolution picture at the second spatial resolution.