US 11,809,975 B2
System and method for end-to-end-differentiable joint image refinement and perception
Felix Heide, Palo Alto, CA (US)
Assigned to Torc CND Robotics, Inc., Montreal (CA)
Filed by TORC CND ROBOTICS, INC., Montreal (CA)
Filed on Jun. 27, 2022, as Appl. No. 17/850,785.
Application 17/850,785 is a continuation of application No. 17/843,174, filed on Jun. 17, 2022.
Application 17/843,174 is a continuation of application No. 17/712,727, filed on Apr. 4, 2022.
Application 17/712,727 is a continuation of application No. 16/927,741, filed on Jul. 13, 2020, granted, now 11,295,176, issued on Apr. 5, 2022.
Application 16/927,741 is a continuation of application No. 16/025,776, filed on Jul. 2, 2018, granted, now 10,713,537, issued on Jul. 14, 2020.
Claims priority of provisional application 62/528,054, filed on Jul. 1, 2017.
Prior Publication US 2022/0335261 A1, Oct. 20, 2022
Int. Cl. G06N 20/00 (2019.01); G06T 5/00 (2006.01); G06T 5/50 (2006.01); G06N 3/084 (2023.01); G06V 10/44 (2022.01); G06F 18/241 (2023.01); G06F 18/2413 (2023.01); G06N 3/045 (2023.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01)
CPC G06N 20/00 (2019.01) [G06F 18/241 (2023.01); G06F 18/24133 (2023.01); G06N 3/045 (2023.01); G06N 3/084 (2013.01); G06T 5/001 (2013.01); G06T 5/003 (2013.01); G06T 5/50 (2013.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20182 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A system for end-to-end differentiable joint image refinement and perception, comprising:
a processor;
a learning machine, having a memory having computer readable instructions stored thereon for execution by the processor, causing the processor to:
determine a representation of a raw image of said plurality of raw images;
initialize a plurality of representation parameters of said representation;
define a plurality of analysis parameters of an image analysis network configured to process said representation; and
jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, comprising joint optimization of nested bilevel objective functions, thereby enabling formulation of said combined objective function as an outer objective function relevant to said image analysis network and an inner objective function relevant to said representation.