US 12,110,041 B2
System and method for completing trajectory prediction from agent-augmented environments
Chiho Choi, San Jose, CA (US); Srikanth Malla, Sunnyvale, CA (US); and Sangjae Bae, Berkley, CA (US)
Assigned to HONDA MOTOR CO., LTD., Tokyo (JP)
Filed by Honda Motor Co., Ltd., Tokyo (JP)
Filed on Jan. 28, 2021, as Appl. No. 17/161,136.
Claims priority of provisional application 63/113,710, filed on Nov. 13, 2020.
Prior Publication US 2022/0153307 A1, May 19, 2022
Int. Cl. B60W 60/00 (2020.01); G06V 20/56 (2022.01)
CPC B60W 60/0027 (2020.02) [G06V 20/56 (2022.01); B60W 2420/403 (2013.01); B60W 2554/20 (2020.02); B60W 2554/404 (2020.02)] 17 Claims
OG exemplary drawing
 
1. A computer-implemented method for completing trajectory prediction from agent-augmented environments, comprising:
receiving image data associated with surrounding environment of an ego agent, wherein the image data includes dynamic agents and static structures located within the surrounding environment of the ego agent;
processing an agent-augmented static representation which is an augmented representation of the surrounding environment of the ego agent based on the image data, including augmenting boundaries of the static structures with respective augmented static agents that represent an obstruction to respective prospective paths of the dynamic agents that are caused by the existence of the respective static structures, further including representing at least one of the static structures as multiple augmented static agents enabling learning of physical constraints of the surrounding environment of the ego agent and predicting implausible respective future paths of the dynamic agents in view of the physical constraints of the surrounding environment;
processing the agent-augmented static representation of the surrounding environment of the ego agent includes augmenting boundaries of the static agents with augmented static agents that represent an occlusion to respective paths of the dynamic agents that are caused by the static structures, and leveraging a similarity between at least one static agent and at least one static structures to represent environmental structures as multiple augmented static agents to further enable learning of physical constraints of the surrounding environment of the ego agent;
processing a set of spatial graphs that correspond to an observation time horizon based on the agent-augmented static representation, wherein nodes in the spatial graphs are simultaneously clustered considering their spatial and temporal interactions;
predicting future trajectories of agents that are located within the surrounding environment of the ego agent based on the spatial graphs; and
controlling at least one component of the ego agent based on the predicted future trajectories of agents within the surrounding environment of the ego agent to avoid overlap with the respective predicted future trajectories of the agents.