US 12,236,692 B2
Driver attention detection method
Yitian Wu, Plano, TX (US); Fatih Murat Porikli, Plano, TX (US); Lei Yang, Plano, TX (US); and Yuan Ma, Plano, TX (US)
Assigned to Shenzhen Yinwang Intelligent Technologies Co., Ltd., Shenzhen (CN)
Filed by SHENZHEN YINWANG INTELLIGENT TECHNOLOGIES CO., LTD., Shenzhen (CN)
Filed on Jun. 3, 2021, as Appl. No. 17/338,390.
Application 17/338,390 is a continuation of application No. PCT/US2019/036366, filed on Jun. 10, 2019.
Prior Publication US 2021/0357670 A1, Nov. 18, 2021
Int. Cl. G06V 20/59 (2022.01); G06T 7/70 (2017.01); G06V 10/82 (2022.01); G06V 20/58 (2022.01)
CPC G06V 20/597 (2022.01) [G06T 7/70 (2017.01); G06V 10/82 (2022.01); G06V 20/58 (2022.01); G06T 2207/20084 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for monitoring driver attentiveness in a vehicle, comprising:
collecting vehicle data and scene information from the vehicle while traveling on a route;
capturing a gaze of a driver to track a gaze direction and a duration of the driver while driving the vehicle on the route;
generating a reference heat map from the scene information and the vehicle data using a convolutional neural network (CNN) that comprises one or more convolutional layers and at least one fully-connected layer, the reference heat map identifying one or more regions in the scene information requiring driver attention for safe driving;
generating a driver gaze heat map based on the gaze direction and the duration of the driver while driving the vehicle using the CNN, the driver gaze heat map identifying one or more zones in the scene information viewed by the driver during the duration;
encoding, by a first subnetwork of a deep learning network, the reference heat map to generate a feature vector R:
encoding, by a second subnetwork of the deep learning network, the driver gaze heat map to generate a feature vector G;
determining a level of driver distraction of the driver in the vehicle based on a distance between the feature vector R and the feature vector G;
merging the feature vector R and the feature vector G to form a driver attention heat map, wherein the deep learning network determines whether the level of driver distraction meets or exceeds a threshold attentiveness level based on the driver attention heat map and generates driver attention results; and
outputting a recommendation to the driver of the vehicle according to the driver attention results.