US 12,293,543 B1
Computer vision techniques using projections of map data into image data
Xin Wang, Sunnyvale, CA (US); and Xinyu Xu, San Jose, CA (US)
Assigned to Zoox, Inc., Foster City, CA (US)
Filed by Zoox, Inc., Foster City, CA (US)
Filed on Dec. 19, 2022, as Appl. No. 18/068,329.
Int. Cl. G06T 7/73 (2017.01); B60W 40/04 (2006.01); B60W 60/00 (2020.01); G01C 21/30 (2006.01); G06T 7/11 (2017.01); G06T 7/50 (2017.01)
CPC G06T 7/74 (2017.01) [B60W 40/04 (2013.01); B60W 60/001 (2020.02); G01C 21/30 (2013.01); G06T 7/11 (2017.01); G06T 7/50 (2017.01); B60W 2420/403 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30252 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:
receiving image data from an image sensor associated with an autonomous vehicle in an environment;
receiving map data associated with the environment, the map data comprising a map object;
projecting, based at least in part on an estimated location of the autonomous vehicle in the map data, the map object into the image data to determine a projection region for the map object in the image data;
determining, based on the estimated location of the autonomous vehicle in the map data, a coordinate channel associated with the projection region;
providing model input data comprising the image data and the coordinate channel to a machine learning model;
receiving, as model output data for the machine learning model, at least one of: (i) a depth map for the image data that comprises depth data for pixels of image data, or (ii) bounding box feature data for an object that is detected within the image data; and
controlling the autonomous vehicle based on the model output data.