US 12,033,334 B2
Neural network for object detection and tracking
Vahid R. Ramezani, Los Altos, CA (US); Akshay Rangesh, San Diego, CA (US); Benjamin Englard, Palo Alto, CA (US); Siddhesh S. Mhatre, Foster City, CA (US); Meseret R. Gebre, Palo Alto, CA (US); and Pranav Maheshwari, Palo Alto, CA (US)
Assigned to Luminar Technologies, Inc., Orlando, FL (US)
Filed by Luminar Technologies, Inc., Orlando, FL (US)
Filed on Jun. 13, 2022, as Appl. No. 17/839,448.
Application 17/839,448 is a continuation of application No. 17/013,446, filed on Sep. 4, 2020, granted, now 11,361,449.
Claims priority of provisional application 63/021,087, filed on May 6, 2020.
Prior Publication US 2022/0309685 A1, Sep. 29, 2022
Int. Cl. G06T 7/246 (2017.01); G06N 3/044 (2023.01); G06N 3/08 (2023.01); G06T 7/73 (2017.01)
CPC G06T 7/246 (2017.01) [G06N 3/044 (2023.01); G06N 3/08 (2013.01); G06T 7/73 (2017.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of multi-object tracking, the method comprising:
receiving, by processing hardware, a sequence of images generated at respective times by one or more sensors configured to sense an environment through which objects are moving relative to the one or more sensors;
constructing, by the processing hardware, a message passing graph having a multiplicity of layers associated with the sequence of images, the constructing including:
generating a plurality of feature nodes to represent features detected in at least a portion of the sequence of images, and
generating edges that interconnect at least some of the feature nodes across adjacent layers of the message passing graph to represent associations between the features;
training a neural network supported by the message passing graph, the training including: performing a pass through the message passing graph in a forward direction including by adding a new feature node based on a feature detection and a new edge node, and performing a pass through the message passing graph in a backward direction, including by updating at least one edge node of the message passing graph; and
tracking, by the processing hardware, multiple features through the sequence of images, including passing messages through the message passing graph.