CPC G05D 1/0253 (2013.01) [G01S 17/89 (2013.01); G05D 1/0088 (2013.01); G05D 1/0219 (2013.01); G05D 1/0221 (2013.01); G05D 1/0248 (2013.01); G06N 20/00 (2019.01); G05D 2201/0213 (2013.01)] | 7 Claims |
1. A computer-implemented method for trajectory prediction and motion device control, comprising:
receiving a first set of data about a surrounding environment of a motion device, wherein the first set of data is LiDAR point cloud data representing a past sequence of top-down views and first position data from the motion device;
receiving a second set of data about the surrounding environment of the motion device, wherein the second set of data is image data representing a past sequence of frontal views and second position data from the motion device, wherein the second set of data is a different modality than the first set of data;
extracting features from the first set of data that model motion behavior of the motion device;
extracting features from the second set of data that model motion behavior of the motion device;
inputting the features from the first set of data and the features from the second set of data into a machine learning model with a shared embedded space;
computing a first trajectory based on a first latent variable sampled from the shared embedded space and a second trajectory based on a second latent variable sampled from the shared embedded space, wherein the first latent variable has the same modality as the first set of data and the second latent variable has the same modality as the second set of data; and
controlling the motion device based on the first trajectory and the second trajectory.
|