US 11,895,409 B2
Image processing based on object categorization
Eran Pinhasov, Zichron Yaakov (IL); Scott Cheng, Foothill Ranch, CA (US); Eran Scharam, Misgav (IL); and Anatoly Gurevich, Haifa (IL)
Assigned to QUALCOMM Incorporated, San Diego, CA (US)
Filed by QUALCOMM Incorporated, San Diego, CA (US)
Filed on Jan. 26, 2021, as Appl. No. 17/158,917.
Claims priority of provisional application 63/068,333, filed on Aug. 20, 2020.
Prior Publication US 2022/0060619 A1, Feb. 24, 2022
Int. Cl. H04N 23/61 (2023.01); H04N 23/80 (2023.01); G06V 20/10 (2022.01); G06V 20/00 (2022.01); H04N 1/60 (2006.01); G06V 10/10 (2022.01); H04N 23/66 (2023.01); H04N 23/70 (2023.01); H04N 101/00 (2006.01); H04N 23/88 (2023.01)
CPC H04N 23/80 (2023.01) [G06V 10/10 (2022.01); G06V 20/10 (2022.01); G06V 20/35 (2022.01); H04N 1/6072 (2013.01); H04N 1/6083 (2013.01); H04N 23/61 (2023.01); H04N 23/66 (2023.01); H04N 23/70 (2023.01); H04N 23/88 (2023.01); H04N 2101/00 (2013.01)] 32 Claims
OG exemplary drawing
 
1. An apparatus for image processing, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor configured to:
receive image data captured by an image sensor;
determine that a first object image region of a plurality of object image regions in the image data depicts a first object category of a plurality of object categories and that a second object image region of the plurality of object image regions in the image data depicts a second object category of the plurality of object categories to generate a category map associated with the image data;
identify a plurality of confidence levels corresponding to a plurality of confidence image regions of a confidence map associated with the image data, wherein each confidence level of the plurality of confidence levels identifies a confidence associated with a categorization in the category map that a corresponding confidence image region of the plurality of confidence image regions depicts one of the plurality of object categories;
generate, based on the category map and the confidence map, a plurality of modifiers for a predetermined setting for an image signal processor (ISP) tuning parameter, the plurality of modifiers identifying a first deviation from the predetermined setting for a first intersection, wherein the first intersection is an intersection of the first object image region and a first confidence image region, the first confidence image region corresponding to a first confidence level, the plurality of modifiers identifying a second deviation from the predetermined setting for a second intersection, wherein the second intersection is an intersection of the first object image region and a second confidence image region, the second confidence image region corresponding to a second confidence level, the plurality of modifiers identifying a third deviation from the predetermined setting for a third intersection, wherein the third intersection is an intersection of the second object image region and a third confidence image region; and
generate an image based on the image data using the ISP tuning parameter at least in part by applying different settings for the ISP tuning parameter to different portions of the image data, the different settings for the ISP tuning parameter being based on the plurality of modifiers, the different portions of the image data including a plurality of intersections of different object image regions of the plurality of object image regions and different confidence image regions of the plurality of confidence image regions, the plurality of intersections including at least the first intersection, the second intersection, and the third intersection.