US 11,951,993 B2
Vehicular driver monitoring system with driver attentiveness and heart rate monitoring
Yuesheng Lu, Farmington Hills, MI (US)
Assigned to Magna Electronics Inc., Auburn Hills, MI (US)
Filed by Magna Electronics Inc., Auburn Hills, MI (US)
Filed on Apr. 3, 2023, as Appl. No. 18/194,703.
Application 18/194,703 is a continuation of application No. 17/929,423, filed on Sep. 2, 2022, granted, now 11,618,454.
Application 17/929,423 is a continuation of application No. 16/946,848, filed on Jul. 9, 2020, granted, now 11,433,906, issued on Sep. 6, 2022.
Claims priority of provisional application 62/872,779, filed on Jul. 11, 2019.
Prior Publication US 2023/0242124 A1, Aug. 3, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. H04N 5/44 (2011.01); B60W 40/08 (2012.01); B60W 50/14 (2020.01); G06T 7/73 (2017.01); G06V 20/59 (2022.01); G06V 40/16 (2022.01)
CPC B60W 40/08 (2013.01) [B60W 50/14 (2013.01); G06T 7/74 (2017.01); G06V 20/597 (2022.01); G06V 40/162 (2022.01); B60R 2300/103 (2013.01); B60R 2300/8006 (2013.01); B60W 2040/0872 (2013.01); B60W 2540/221 (2020.02); B60W 2540/229 (2020.02); G06T 2207/30201 (2013.01)] 29 Claims
OG exemplary drawing
 
1. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:
a camera disposed at an interior rearview mirror assembly of a vehicle equipped with the vehicular driver monitoring system, the camera viewing at least a face of a driver of the vehicle;
wherein the camera comprises a CMOS image sensor having at least one million photosensing elements arranged in rows and columns, and wherein the camera comprises spectral filtering at the CMOS image sensor;
wherein the camera is operable to capture image data;
an electronic control unit (ECU) comprising electronic circuitry and associated software, the electronic circuitry including an image processor that is operable to process image data captured by the camera;
wherein, while the driver is driving the vehicle, the vehicular driver monitoring system detects changes of the driver's eyes via processing at the ECU of image data captured by the camera, and wherein the vehicular driver monitoring system determines driver attentiveness based at least in part on detected changes of the driver's eyes;
wherein the vehicular driver monitoring system comprises a remote Photoplethysmography (rPPG) system for monitoring a heart rate of the driver;
wherein, based at least in part on the determined driver attentiveness, and responsive to monitoring the heart rate of the driver, the vehicular driver monitoring system determines impairment of the driver's capability to operate the vehicle; and
wherein the vehicular driver monitoring system generates an alert responsive to at least one selected from the group consisting of (i) determination, via the remote Photoplethysmography (rPPG) system, that the driver's heart rate changes greater than a threshold change over a predetermined period of time and (ii) determination, via processing at the ECU of image data captured by the camera, that driver attentiveness is below a threshold level.