US 12,307,714 B2
Gaze calibration system
Ryota Nakamura, Tokyo (JP); and Taiyo Matsuhashi, Tokyo (JP)
Assigned to SUBARU CORPORATION, Tokyo (JP)
Filed by SUBARU CORPORATION, Tokyo (JP)
Filed on Nov. 30, 2021, as Appl. No. 17/538,165.
Claims priority of application No. 2020-218516 (JP), filed on Dec. 28, 2020.
Prior Publication US 2022/0207773 A1, Jun. 30, 2022
Int. Cl. G06T 7/80 (2017.01); G06F 3/14 (2006.01); G06T 7/70 (2017.01); G06V 20/59 (2022.01)
CPC G06T 7/80 (2017.01) [G06F 3/1423 (2013.01); G06T 7/70 (2017.01); G06V 20/597 (2022.01); G06T 2207/30201 (2013.01); G06T 2207/30268 (2013.01)] 2 Claims
OG exemplary drawing
 
1. A gaze calibration system to be applied to a vehicle, the gaze calibration system comprising:
a first display unit disposed at a center in a widthwise direction of the vehicle and in front of a driver's seat of the vehicle;
a second display unit disposed directly facing the driver's seat, wherein the second display unit is a head-up display;
a camera disposed on an upper portion of an outer periphery of the first display unit and configured to capture an image of a driver seated on the driver's seat, the image including a face of the driver; and
a processor configured to:
monitor a driving state of the driver based on where the driver gazes, using the image of the driver captured by the camera;
control a display image on the first display unit and a display image on the second display unit;
display, on the first display unit, a personal registration screen for registering the driver as a target for monitoring the driving state;
in response to displaying the personal registration screen on the first display unit, acquire a first facial image of the driver from the image captured by the camera while displaying the personal registration screen on the first display unit, wherein the first facial image of the driver is captured in vicinity of an optical axis of the imaging unit;
in response to acquiring the first facial image, (1) register the driver as the target based on the first facial image and (2) display, across an entire screen of the second display unit, a sequential moving image in which a marker to be gazed at by the driver is displayed while making successive transitions;
in response to displaying the sequential moving image on the second display unit, acquire a second facial image of the driver from the image captured by the camera while displaying the sequential moving image on the second display unit, wherein the second facial image is captured off the optical axis of the imaging unit;
in response to acquiring the second facial image, detect, from the first and second facial images, a first facial pose and a first gaze direction of the driver when the driver gazes at the first display unit, and a second facial pose and a second gaze direction of the driver when the driver gazes at the second display unit, to generate calibration data; and
perform calibration of a gaze direction corresponding to a reference facial pose when the driver gazes at the first display unit, and a gaze direction corresponding to a reference facial pose when the driver gazes at the second display unit, on a basis of the calibration data.