US 11,893,801 B2
Flagman traffic gesture recognition
Eran Kishon, Hod Hasharon (IL); Weijing Shi, Pittsburgh, PA (US); and Ragunathan Rajkumar, Monroeville, PA (US)
Assigned to GM GLOBAL TECHNOLOGY OPERATIONS LLC, Detroit, MI (US)
Filed by GM Global Technology Operations LLC, Detroit, MI (US); and Carnegie Mellon University, Pittsburgh, PA (US)
Filed on Apr. 6, 2021, as Appl. No. 17/223,473.
Prior Publication US 2022/0318560 A1, Oct. 6, 2022
Int. Cl. G06V 20/58 (2022.01); G06N 3/04 (2023.01); G06T 11/20 (2006.01); G06V 40/20 (2022.01); G06F 18/2431 (2023.01); A23B 7/159 (2006.01); B01D 53/04 (2006.01); B01D 53/22 (2006.01); B01D 53/26 (2006.01); F25D 17/04 (2006.01); F25D 25/02 (2006.01)
CPC G06V 20/582 (2022.01) [A23B 7/159 (2013.01); B01D 53/0415 (2013.01); B01D 53/229 (2013.01); B01D 53/268 (2013.01); F25D 17/042 (2013.01); F25D 25/025 (2013.01); G06F 18/2431 (2023.01); G06N 3/04 (2013.01); G06T 11/20 (2013.01); G06V 40/20 (2022.01); A23V 2002/00 (2013.01); B01D 2257/7022 (2013.01); B01D 2257/80 (2013.01); F25D 2317/041 (2013.01); F25D 2325/00 (2013.01); G06T 2210/12 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method of operating a vehicle based on a gesture made by a traffic director, comprising:
inputting an image of the traffic director into at least one neural network, wherein the traffic director is using a traffic prop;
generating a skeleton of the traffic direction from the image;
locating a hand of the traffic director in the image from a ray drawn through an elbow key point of the skeleton and a wrist key point of the skeleton;
locating a bounding box in the image that indicates the traffic prop;
generating, at the at least one neural network, an encoded hand vector based on a configuration of the hand of the traffic director from the image;
combining, at the at least one neural network, the skeleton of the traffic director in the bounding box, and the encoded hand vector to generate a representation vector;
predicting, at the at least one neural network, the gesture from the representation vector; and
operating the vehicle based on the meaning of the gesture.