US 12,361,569 B2
Deep equilibrium flow estimation
Shaojie Bai, Pittsburgh, PA (US); Yash Savani, Pittsburgh, PA (US); Jeremy Kolter, Pittsburgh, PA (US); Devin T. Willmott, Pittsburgh, PA (US); João D. Semedo, Pittsburgh, PA (US); and Filipe Condessa, Pittsburgh, PA (US)
Assigned to Robert Bosch GmbH, Stuttgart (DE)
Filed by Robert Bosch GmbH, Stuttgart (DE)
Filed on Mar. 28, 2022, as Appl. No. 17/706,064.
Prior Publication US 2023/0306617 A1, Sep. 28, 2023
Int. Cl. G06T 7/269 (2017.01)
CPC G06T 7/269 (2017.01) [G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30252 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A computer-implemented method for a machine learning (ML) system comprising:
receiving a first image frame and a second image frame from a sensor, wherein the first image frame and the second image frame are time series data;
determining a first flow state and a first latent state of the first image frame;
determining a Deep Equilibrium Model (DEQ) based fixed-point solution via a root finding method based on the first flow state, the first latent state, and a layer function to obtain an estimated flow state (f) and an estimated latent state (h), the fixed-point solution (z*) represented by the following equation:
z*=(h*,f),
where h* is a fixed-point flow state and f* is a fixed-point latent state;
receiving a third image frame, wherein the second image frame and the third image frame are time series data;
determining the DEQ based fix-fixed-point solution via the root finding method based on the estimated flow state, the estimated latent state, and the layer function to obtain an updated flow state and updated latent state, the layer function is updated based on a total loss (custom charactertotal) comprised of a main loss (custom charactermain) and a correction loss (custom charactercor) and represented by the following equation:

OG Complex Work Unit Math
where fgt is a ground truth optical flow, γ is less than one and is a loss weight hyperparameter, and f[i] is part of randomly picked z[i]=(h[i], f[i]) on a convergence path (z[0], . . . ,z[i], . . . z*), where z[0] is an initial guess; and
outputting the updated flow state.