US 12,080,161 B2
Traffic signal understandings and representation for prediction, planning, and control
Kuan-Hui Lee, San Francisco, CA (US); Charles Christopher Ochoa, San Francisco, CA (US); Arjun Bhargava, San Francisco, CA (US); Chao Fang, Sunnyvale, CA (US); and Kun-Hsin Chen, San Francisco, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US); and TOYOTA JIDOSHA KABUSHIKI KAISHA, Aichi-Ken (JP)
Filed by TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed on Apr. 28, 2022, as Appl. No. 17/732,376.
Prior Publication US 2023/0351886 A1, Nov. 2, 2023
Int. Cl. G08G 1/01 (2006.01)
CPC G08G 1/0125 (2013.01) [G08G 1/0141 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for vehicle prediction, planning, and control, comprising:
recording, using a monocular camera of an ego vehicle, visual traffic signal state information at an intersection, visual turn light and/or brake light state information of surrounding autonomous dynamic objects (ADOs), and traffic state information of the ego vehicle;
separately encoding the visual traffic signal state information, the visual turn light and/or brake light state information, and the traffic state information into corresponding traffic state latent spaces;
aggregating the corresponding traffic state latent spaces to form a generalized traffic geometry latent space;
interpreting the generalized traffic geometry latent space to form a traffic flow map including current and future vehicle trajectories;
decoding the generalized traffic geometry latent space to predict a vehicle behavior according to the traffic flow map based on the current and future vehicle trajectories; and
controlling the ego vehicle to follow a planned trajectory at the intersection in response to the predicted vehicle behavior and the current and future vehicle trajectories based on the traffic flow map without relying on map data or light detection and ranging (LIDAR) or radio detection and ranging (RADAR) sensors.