US 12,441,301 B2
Sequential pedestrian trajectory prediction using step attention for collision avoidance
Neda Masoud, Ann Arbor, MI (US); Mahdi Bandegi, Ann Arbor, MI (US); Joseph Lull, South Haven, MI (US); Rajesh Malhan, Troy, MI (US); and Ethan Zhang, Ann Arbor, MI (US)
Assigned to DENSO International America, Inc, Southfield, MI (US); and The Regents of The University of Michigan, Ann Arbor, MI (US)
Filed by DENSO International America, Inc., Southfield, MI (US); and THE REGENTS OF THE UNIVERSITY OF MICHIGAN, Ann Arbor, MI (US)
Filed on Aug. 3, 2022, as Appl. No. 17/879,966.
Claims priority of provisional application 63/229,202, filed on Aug. 4, 2021.
Prior Publication US 2023/0038673 A1, Feb. 9, 2023
Int. Cl. B60W 30/09 (2012.01); B60W 30/095 (2012.01); B60W 40/04 (2006.01); B60W 50/00 (2006.01); B60W 60/00 (2020.01); G06N 3/04 (2023.01)
CPC B60W 30/09 (2013.01) [B60W 30/0956 (2013.01); B60W 40/04 (2013.01); B60W 50/0097 (2013.01); B60W 60/0011 (2020.02); B60W 60/0015 (2020.02); B60W 60/00274 (2020.02); G06N 3/04 (2013.01); B60W 2050/0005 (2013.01); B60W 2554/4029 (2020.02); B60W 2554/4041 (2020.02)] 32 Claims
OG exemplary drawing
 
1. A pedestrian tracking system comprising:
a buffer or a memory configured to store a trajectory sequence of a pedestrian;
a step attention module configured to iteratively perform a step attention process to predict a plurality of states of the pedestrian and iteratively modify the stored trajectory sequence to provide a resultant trajectory sequence including the predicted plurality of states, each iteration of the step attention process comprising the step attention module
learning the stored trajectory sequence to provide a plurality of time-dependent hidden states,
reshaping each of the plurality of time-dependent hidden states to provide two-dimensional tensors,
condensing the two-dimensional tensors via a plurality of convolutional networks to provide convolutional sequences for each of a plurality of time steps,
learning hidden feature patterns within each of the plurality of time steps via the plurality of convolutional networks based on the convolution sequences,
introducing relative spatial logits to represent relative position information of inputs of the plurality of convolution networks,
based on the hidden feature patterns and the relative position information of the inputs, learn global patterns of the inputs for each of the plurality of time steps,
capturing global information of the convolutional sequences including the global patterns to output a set of trajectory patterns represented by a new sequence of tensors,
learning time-related patterns in the new sequence of tensors and decoding the new sequence to provide one or more of the predicted plurality of states of the pedestrian, and
modifying the stored trajectory sequence to include the one or more of the predicted plurality of states; and
a control module configured to, based on the resultant trajectory sequence, perform a countermeasure operation to avoid a collision with the pedestrian.