US 12,094,222 B2
Cabin monitoring and situation understanding perceiving method and system thereof
Yi-Jie Lin, Changhua County (TW)
Assigned to Automotive Research & Testing Center, Changhua County (TW)
Filed by Automotive Research & Testing Center, Changhua County (TW)
Filed on Nov. 30, 2021, as Appl. No. 17/457,030.
Prior Publication US 2023/0169782 A1, Jun. 1, 2023
Int. Cl. G06V 20/00 (2022.01); B60W 40/08 (2012.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G06T 7/70 (2017.01); G06V 10/60 (2022.01); G06V 10/80 (2022.01); G06V 10/82 (2022.01); G06V 20/58 (2022.01); G06V 20/59 (2022.01); G06V 40/16 (2022.01); G06V 40/20 (2022.01)
CPC G06V 20/59 (2022.01) [B60W 40/08 (2013.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06T 7/70 (2017.01); G06V 10/60 (2022.01); G06V 10/80 (2022.01); G06V 10/82 (2022.01); G06V 20/58 (2022.01); G06V 40/165 (2022.01); G06V 40/171 (2022.01); G06V 40/172 (2022.01); G06V 40/20 (2022.01); B60W 2040/089 (2013.01); B60W 2540/21 (2020.02); G06T 2207/10144 (2013.01); G06T 2207/10152 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20221 (2013.01); G06T 2207/30201 (2013.01); G06T 2207/30268 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A cabin monitoring and situation understanding perceiving method, which is configured to monitor an inside vehicle cabin and perceive a situation understanding result, the cabin monitoring and situation understanding perceiving method comprising:
performing a cabin interior image capturing step to drive a first camera unit to capture a cabin interior image from the inside vehicle cabin and store the cabin interior image to a storage unit, wherein the cabin interior image comprises an overexposed image, a normally exposed image and an underexposed image;
performing a generative adversarial network model creating step to drive a processing unit to create a generative adversarial network model according to the overexposed image, the normally exposed image and the underexposed image of the cabin interior image from the storage unit;
performing an image adjusting step to drive the processing unit to adjust the cabin interior image according to the generative adversarial network model to generate an approximate image approximating the normally exposed image;
performing a cabin interior monitoring step to drive the processing unit to process the approximate image according to a facial recognizing model and a human pose estimating model to generate a facial recognizing result and a human pose estimating result;
performing a cabin exterior image and voice capturing step to drive a second camera unit to capture a cabin exterior image and store the cabin exterior image to the storage unit, and drive an audio unit to capture a voice information from the inside vehicle cabin and store the voice information to the storage unit; and
performing a situation understanding perceiving step to drive the processing unit to process at least one of the approximate image, the cabin exterior image and the voice information according to a situation understanding model to perceive the situation understanding result;
wherein the storage unit accesses an allowable value, and the image adjusting step comprises:
performing a luminosity confirming step to drive the processing unit to perform an evaluation operating procedure on the approximate image and the normally exposed image, wherein the evaluation operating procedure confirms whether a luminosity difference between the approximate image and the normally exposed image is within the allowable value.