CPC G06V 20/582 (2022.01) [A23B 7/159 (2013.01); B01D 53/0415 (2013.01); B01D 53/229 (2013.01); B01D 53/268 (2013.01); F25D 17/042 (2013.01); F25D 25/025 (2013.01); G06F 18/2431 (2023.01); G06N 3/04 (2013.01); G06T 11/20 (2013.01); G06V 40/20 (2022.01); A23V 2002/00 (2013.01); B01D 2257/7022 (2013.01); B01D 2257/80 (2013.01); F25D 2317/041 (2013.01); F25D 2325/00 (2013.01); G06T 2210/12 (2013.01)] | 14 Claims |
1. A method of operating a vehicle based on a gesture made by a traffic director, comprising:
inputting an image of the traffic director into at least one neural network, wherein the traffic director is using a traffic prop;
generating a skeleton of the traffic direction from the image;
locating a hand of the traffic director in the image from a ray drawn through an elbow key point of the skeleton and a wrist key point of the skeleton;
locating a bounding box in the image that indicates the traffic prop;
generating, at the at least one neural network, an encoded hand vector based on a configuration of the hand of the traffic director from the image;
combining, at the at least one neural network, the skeleton of the traffic director in the bounding box, and the encoded hand vector to generate a representation vector;
predicting, at the at least one neural network, the gesture from the representation vector; and
operating the vehicle based on the meaning of the gesture.
|