US 12,258,047 B2
System and method for providing long term and key intentions for trajectory prediction
Harshayu Girase, Union City, CA (US); Haiming Gang, San Jose, CA (US); Srikanth Malla, Sunnyvale, CA (US); Jiachen Li, Albany, CA (US); Akira Kanehara, Utsunomiya (JP); and Chiho Choi, San Jose, CA (US)
Assigned to Honda Motor Co., Ltd., Tokyo (JP)
Filed by Honda Motor Co., Ltd., Tokyo (JP)
Filed on Jun. 21, 2021, as Appl. No. 17/352,540.
Claims priority of provisional application 63/166,195, filed on Mar. 25, 2021.
Prior Publication US 2022/0306160 A1, Sep. 29, 2022
Int. Cl. B60W 60/00 (2020.01); G01S 17/86 (2020.01); G01S 17/894 (2020.01); G06T 7/20 (2017.01)
CPC B60W 60/0027 (2020.02) [G01S 17/86 (2020.01); G01S 17/894 (2020.01); G06T 7/20 (2013.01); B60W 2420/403 (2013.01); B60W 2420/408 (2024.01); B60W 2554/4029 (2020.02); B60W 2554/4045 (2020.02); B60W 2556/10 (2020.02); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20072 (2013.01); G06T 2207/30196 (2013.01); G06T 2207/30236 (2013.01); G06T 2207/30241 (2013.01); G06T 2207/30252 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for providing long term and key intentions for trajectory prediction, comprising:
receiving and aggregating image data and LiDAR data associated with RGB images and LiDAR point clouds that are associated with a surrounding environment of an ego agent, wherein
the aggregated image data and LiDAR data is aggregated environment data;
processing a long term and key intentions for trajectory prediction dataset (LOKI dataset) that is utilized to complete joint trajectory and intention prediction for heterogeneous traffic agents, wherein the LOKI dataset is populated with annotations that include the aggregated environment data and annotated labels that pertain to attributes that influence agent intent for each of the heterogeneous traffic agents;
encoding a past observation history of each of the heterogeneous traffic agents and sampling a respective goal; and
decoding and predicting future trajectories associated with each of the heterogeneous traffic agents based on data included within the LOKI dataset, the encoded past observation history, and the respective goal, wherein
the annotated labels include contextual labels that are associated with factors that affect future behavior of each of the heterogeneous traffic agents including at least one of weather and road conditions,
a scene graph derived from the LOKI dataset is constructed that includes nodes including agent trajectory information, agent intent information, and goal information of the heterogenous traffic agents located within the surrounding environment of the ego agent, and nodes denoting road entrance and road exit information which provide the heterogeneous traffic agents with map topology information pertaining to the surrounding environment, and
decoding and predicting future trajectories includes a decoder being configured to access the LOKI dataset to analyze the LOKI dataset in addition to the scene graph to determine the predicted intentions, goals, and past motion of all of the heterogenous traffic agents and a scene of the surrounding environment of the ego agent to predict the trajectories of each of the heterogenous traffic agents that are located within the surrounding environment of the ego agent.