US 12,361,567 B1
Multi-frame temporal aggregation and dense motion estimation for autonomous vehicles
Daniel Rudolf Maurer, Mountain View, CA (US); Kratarth Goel, Fremont, CA (US); Alper Ayvaci, San Jose, CA (US); Vasiliy Igorevich Karasev, San Francisco, CA (US); and Hang Yan, Sunnyvale, CA (US)
Assigned to Waymo LLC, Mountain View, CA (US)
Filed by Waymo LLC, Mountain View, CA (US)
Filed on Dec. 2, 2022, as Appl. No. 18/073,996.
Int. Cl. G06T 7/246 (2017.01); B60W 60/00 (2020.01); G06T 3/18 (2024.01)
CPC G06T 7/248 (2017.01) [B60W 60/001 (2020.02); G06T 3/18 (2024.01); B60W 2420/403 (2013.01); B60W 2420/408 (2024.01); B60W 2554/404 (2020.02); G06T 2207/10028 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/30261 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining, by a processing device, input data derived from a set of sensors associated with an autonomous vehicle (AV), wherein the input data comprises camera data and radar data comprising a plurality of frames each associated with a timestep;
extracting, by the processing device from the input data, a plurality of sets of bird's-eye view (BEV) features, wherein each set of BEV features corresponds to a respective timestep associated with a respective frame;
generating, by the processing device from the plurality of sets of BEV features, an object flow for at least one object, wherein generating the object flow comprises performing at least one of:
multi-frame temporal aggregation of sets of BEV features to aggregate at least a first set of BEV features corresponding to a current timestep associated with a first frame and a second set of BEV features corresponding to a prior timestep associated with a second frame, or
multi-frame dense motion estimation for identification of at least one object of an unknown class; and
causing, by the processing device, a driving path of the AV to be modified in view of the object flow.