US 11,808,858 B2
Systems and methods for constructing and utilizing field-of-view (FOV) information
Pingfan Meng, Dublin, CA (US); Yubo Zhang, Los Gatos, CA (US); Wei-Yang Chiu, Fremont, CA (US); and Stephen Lee, Sunnyvale, CA (US)
Assigned to Pony AI Inc., Grand Cayman (KY)
Filed by Pony AI Inc., Grand Caymans (KY)
Filed on Sep. 4, 2019, as Appl. No. 16/560,910.
Prior Publication US 2021/0064943 A1, Mar. 4, 2021
Int. Cl. G06F 16/00 (2019.01); G06F 7/00 (2006.01); G01S 17/931 (2020.01); G06F 16/29 (2019.01); G01S 17/89 (2020.01); G06F 16/172 (2019.01); G06V 30/262 (2022.01); G06V 20/56 (2022.01); G06V 10/80 (2022.01)
CPC G01S 17/931 (2020.01) [G01S 17/89 (2013.01); G06F 16/172 (2019.01); G06F 16/29 (2019.01); G06V 10/803 (2022.01); G06V 10/806 (2022.01); G06V 20/56 (2022.01); G06V 30/274 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A method for constructing and utilizing field-of-view (FOV) information, comprising:
receiving a plurality of sensor data input streams from a plurality of sensors associated with a vehicle;
retrieving a subset of pre-populated map data from a slower-access storage and transferring the subset to a faster-access storage, wherein the faster-access storage has a relatively higher data access rate;
performing one or more data transformation operations on the sensor data input streams and the pre-populated data to obtain transformed data;
extracting FOV semantics data from the transformed data, wherein the FOV semantics data comprises spatial and temporal data; and
managing data and execution patterns utilizing the FOV semantics data, wherein the managing comprises:
receiving, from different cores, requests for portions of the subset of the pre-populated map data, wherein each of the different cores requests a different portion of the subset of the pre-populated map data;
allocating a computation of different frames of a first sensor scan to a common core based on a first reduction in latency resulting from retrieving motion compensation data corresponding to different frames of the first sensor scan from a common buffer of the common core, compared to retrieving the motion compensation data from different cores, wherein the motion compensation data corresponding to the different frames of the first sensor scan adjusts for changes in frames of reference of the sensor data input streams caused by road surface conditions and vibrations;
allocating a computation of the first sensor scan and a second sensor scan to different cores based on a second reduction in latency from simultaneously computing different portions of the sensor data input streams, compared to computing only a single portion of the sensor data input streams at any specific time instance;
in response to allocating the computation of the first sensor scan and the second sensor scan to different cores, storing motion compensation data corresponding to the first sensor scan and the second sensor scan in each of the different cores; and
in response to allocating the computation of the different frames to the common core, storing the motion compensation data corresponding to the different frames of the first sensor scan in the common buffer.