US 12,380,058 B2
Hardware-based image/video processing in machine learning-accelerator system-on-chip
Sandeep Pande, Bengaluru (IN); Satish Singh, Bangalore (IN); Colin Beaton Verrilli, Apex, NC (US); Natarajan Vaidhyanathan, Carrboro, NC (US); and Vinay Murthy, Poway, CA (US)
Assigned to QUALCOMM Incorporated, San Diego, CA (US)
Filed by QUALCOMM Incorporated, San Diego, CA (US)
Filed on Jun. 12, 2023, as Appl. No. 18/333,377.
Prior Publication US 2024/0411718 A1, Dec. 12, 2024
Int. Cl. G06F 15/80 (2006.01); G06F 9/50 (2006.01)
CPC G06F 15/80 (2013.01) [G06F 9/5027 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A machine learning (ML)-accelerator system-on-chip (SoC), comprising:
a plurality of ML-accelerator cores;
a network-on-chip (NoC) coupled to the plurality of ML-accelerator cores;
an inference video post processing (infVPP) module coupled to the NoC; and
an on-chip video decoder coupled to the NoC to enable single chip video or imaging ML applications,
wherein the video decoder is operable to decode video/image data received in a first storage format and to store decoded video/image data in a predetermined video format in a first memory of the ML-accelerator SoC,
wherein the infVPP module is operable to convert the decoded video/image data stored in the first memory into activation input data stored in a second memory of the ML-accelerator SoC,
wherein the infVPP module is further operable to provide image processing, image data scaling, cropping, normalizing, data packing, and formatting of the decoded video/image data in a packed dense depth data dump (D4) format to provide the activation input data, and
wherein the infVPP module improves decoding processing efficiency and overall performance of the ML-accelerator cores in terms of inference per second (inference/sec).