US 12,311,925 B2
Divide-and-conquer for lane-aware diverse trajectory prediction
Sriram Nochur Narayanan, San Jose, CA (US); Ramin Moslemi, Pleasanton, CA (US); Francesco Pittaluga, Los Angeles, CA (US); Buyu Liu, Cupertino, CA (US); and Manmohan Chandraker, Santa Clara, CA (US)
Assigned to NEC Corporation, Tokyo (JP)
Filed by NEC Laboratories America, Inc., Princeton, NJ (US)
Filed on Nov. 8, 2021, as Appl. No. 17/521,139.
Claims priority of provisional application 63/113,434, filed on Nov. 13, 2020.
Claims priority of provisional application 63/111,674, filed on Nov. 10, 2020.
Prior Publication US 2022/0144256 A1, May 12, 2022
Int. Cl. B60W 30/095 (2012.01); G06F 16/29 (2019.01); G06N 3/08 (2023.01)
CPC B60W 30/0956 (2013.01) [G06F 16/29 (2019.01); G06N 3/08 (2013.01); B60W 2552/53 (2020.02)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method for driving path prediction and current trajectory control by an advanced driver-assistance system (ADAS) that is integrated into an autonomous vehicle, comprising:
obtaining a top view map, a past trajectory, and lane centerlines for the autonomous vehicle in a training scene as initial training inputs;
ranking the lane centerlines based on heuristics including trajectory distance along a lane score and a centerline yaw score;
concatenating past trajectory features and lane centerline features in a channel dimension at an agent's respective location in the top view map of the training scene to obtain concatenated features thereat;
obtaining, by a convolutional encoder of the ADAS in a single forward pass, convolutional features derived from the top view map, the concatenated features, and a single representation of the training scene that includes the vehicle and interactions with agents in the training scene;
extracting, by a hypercolumn trajectory encoder of the ADAS, hypercolumn descriptor vectors from the convolutional features, the hypercolumn descriptor vectors including the convolutional features from the agent's respective location in the top view map and an interpolated location in subsequent lower convolutional layers;
obtaining, by a hypercolumn trajectory decoder of the ADAS, primary and auxiliary trajectory predictions from the hypercolumn descriptor vectors;
generating, by an Inverse Optimal Control (IOC) based ranking module of the ADAS, a respective score for each of the primary and auxiliary trajectory predictions;
training a vehicle trajectory prediction neural network of the ADAS using a reconstruction loss, a regularization loss objective, and an IOC loss objective responsive to the respective score for each of the primary and auxiliary trajectory predictions;
generating a trajectory prediction of the autonomous vehicle using the vehicle trajectory prediction neural network; and
using the ADAS to autonomously control a current trajectory of the autonomous vehicle based on the trajectory prediction, where the ADAS autonomously controls the vehicle using one or more of a steering system, a braking system, and an accelerating system of the vehicle.