US 11,836,893 B2
Online AI super resolution engine and operations
Cheng Lung Jen, Hsinchu (TW); Pei-Kuei Tsung, Hsinchu (TW); Yao-Sheng Wang, Hsinchu (TW); Chih-Wei Chen, Hsinchu (TW); Chih-Wen Goo, Hsinchu (TW); Yu-Cheng Tseng, Hsinchu (TW); Ming-En Shih, Hsinchu (TW); and Kuo-Chiang Lo, Hsinchu (TW)
Assigned to MediaTek Inc., Hsinchu (TW)
Filed by MediaTek Inc., Hsinchu (TW)
Filed on Feb. 4, 2021, as Appl. No. 17/167,356.
Claims priority of provisional application 62/987,960, filed on Mar. 11, 2020.
Prior Publication US 2021/0287340 A1, Sep. 16, 2021
Int. Cl. G06T 3/40 (2006.01); G06N 3/04 (2023.01); H04L 65/75 (2022.01); G06F 18/214 (2023.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 20/40 (2022.01)
CPC G06T 3/4053 (2013.01) [G06F 18/214 (2023.01); G06N 3/04 (2013.01); G06T 3/4046 (2013.01); G06T 3/4076 (2013.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 20/41 (2022.01); H04L 65/75 (2022.05)] 18 Claims
OG exemplary drawing
 
1. A video processing circuit comprising:
an input buffer to receive input low-resolution (LR) frames and high-resolution (HR) frames in a video stream from a video source over a network, wherein the HR frames are received less frequently than the LR frames;
an online adaptation circuit operative to:
form training pairs, each training pair formed by one of the input LR frames and one of the HR frames in the video stream; and
calculate an update to representative features that characterize the input LR frames using the training pairs; and
an artificial intelligence (AI) super-resolution (SR) circuit operative to:
receive the input LR frames from the input buffer and the representative features from the online adaptation circuit;
update an AI model based on the update to the representative features; and
concurrently with the representative features being updated, generate SR frames for display from the input LR frames in the video stream using the AI model, wherein each SR frame has a higher resolution than a corresponding one of the input LR frames.