US 11,687,087 B2
Systems and methods for shared cross-modal trajectory prediction
Chiho Choi, San Jose, CA (US)
Assigned to HONDA MOTOR CO., LTD., Tokyo (JP)
Filed by Honda Motor Co., Ltd., Tokyo (JP)
Filed on Aug. 10, 2020, as Appl. No. 16/989,066.
Claims priority of provisional application 62/988,791, filed on Mar. 12, 2020.
Prior Publication US 2021/0286371 A1, Sep. 16, 2021
Int. Cl. G05D 1/02 (2020.01); G01S 17/89 (2020.01); G05D 1/00 (2006.01); G06N 20/00 (2019.01)
CPC G05D 1/0253 (2013.01) [G01S 17/89 (2013.01); G05D 1/0088 (2013.01); G05D 1/0219 (2013.01); G05D 1/0221 (2013.01); G05D 1/0248 (2013.01); G06N 20/00 (2019.01); G05D 2201/0213 (2013.01)] 7 Claims
OG exemplary drawing
 
1. A computer-implemented method for trajectory prediction and motion device control, comprising:
receiving a first set of data about a surrounding environment of a motion device, wherein the first set of data is LiDAR point cloud data representing a past sequence of top-down views and first position data from the motion device;
receiving a second set of data about the surrounding environment of the motion device, wherein the second set of data is image data representing a past sequence of frontal views and second position data from the motion device, wherein the second set of data is a different modality than the first set of data;
extracting features from the first set of data that model motion behavior of the motion device;
extracting features from the second set of data that model motion behavior of the motion device;
inputting the features from the first set of data and the features from the second set of data into a machine learning model with a shared embedded space;
computing a first trajectory based on a first latent variable sampled from the shared embedded space and a second trajectory based on a second latent variable sampled from the shared embedded space, wherein the first latent variable has the same modality as the first set of data and the second latent variable has the same modality as the second set of data; and
controlling the motion device based on the first trajectory and the second trajectory.