US 12,437,558 B2
End-to-end signalized intersection transition state estimator with scene graphs over semantic keypoints
Kun-Hsin Chen, Mountain View, CA (US); Peiyan Gong, Ann Arbor, MI (US); Sudeep Pillai, Santa Clara, CA (US); Arjun Bhargava, San Francisco, CA (US); Shunsho Kaku, Mountain View, CA (US); Hai Jin, Ann Arbor, MI (US); and Kuan-Hui Lee, Los Altos, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed by TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed on Oct. 29, 2022, as Appl. No. 17/976,761.
Application 17/976,761 is a continuation of application No. 17/177,516, filed on Feb. 17, 2021, granted, now 11,514,685.
Prior Publication US 2023/0047160 A1, Feb. 16, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06V 20/58 (2022.01); B60W 50/00 (2006.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06F 18/2411 (2023.01); G06V 10/44 (2022.01); G06V 20/56 (2022.01)
CPC G06V 20/584 (2022.01) [B60W 50/00 (2013.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06F 18/2411 (2023.01); G06V 10/443 (2022.01); G06V 20/588 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
at least one memory storing machine-executable instructions; and
at least one processor configured to access the at least one memory and execute the machine-executable instructions to:
determine keypoint embeddings and linkage embeddings based on keypoints and linkages detected from an image;
generate a scene graph of the image based on the keypoint embeddings and linkage embeddings; and
output a vehicle control signal based on the scene graph and localization information for a vehicle.
 
14. A method, comprising:
determining, using a machine learning algorithm and based on detected keypoints and detected linkages associated with an image, keypoint embeddings and linkage embeddings;
determining an association between a particular keypoint of the detected keypoints and a current travel lane of a vehicle based on associations between the keypoint embeddings and the linkage embeddings; and
controlling operation of the vehicle based on a control signal generated based on the determined association between the particular keypoint and the current travel lane of the vehicle.