US 11,798,297 B2
Control device, system and method for determining the perceptual load of a visual and dynamic driving scene
Jonas Ambeck-Madsen, Brussels (BE); Ichiro Sakata, Brussels (BE); Nilli Lavie, London (GB); Gabriel J. Brostow, London (GB); Luke Palmer, London (GB); and Alina Bialkowski, London (GB)
Assigned to TOYOTA MOTOR EUROPE NV/SA, Brussels (BE); and UCL BUSINESS PLC, London (GB)
Appl. No. 16/495,160
Filed by TOYOTA MOTOR EUROPE, Brussels (BE); and UCL BUSINESS PLC, London (GB)
PCT Filed Mar. 21, 2017, PCT No. PCT/EP2017/056726
§ 371(c)(1), (2) Date Sep. 18, 2019,
PCT Pub. No. WO2018/171875, PCT Pub. Date Sep. 27, 2018.
Prior Publication US 2020/0143183 A1, May 7, 2020
Int. Cl. G06V 20/59 (2022.01); G06V 20/58 (2022.01); G06V 20/56 (2022.01); G06V 20/64 (2022.01); G06F 18/213 (2023.01); G06F 18/214 (2023.01); G06F 18/241 (2023.01); G06V 10/82 (2022.01); G06V 10/44 (2022.01); G06F 18/25 (2023.01)
CPC G06V 20/597 (2022.01) [G06F 18/213 (2023.01); G06F 18/214 (2023.01); G06F 18/241 (2023.01); G06V 10/454 (2022.01); G06V 10/82 (2022.01); G06V 20/56 (2022.01); G06V 20/58 (2022.01); G06V 20/588 (2022.01); G06V 20/653 (2022.01); G06F 18/253 (2023.01)] 26 Claims
OG exemplary drawing
 
1. A control device for a vehicle for determining a perceptual load of a visual and dynamic driving scene on a driver, the perceptual load on the driver corresponding to an amount of load on a brain of a driver to fully perceive temporal portions and spatial portions of the visual and dynamic driving scene, and the perceptual load being based only on a predetermined load model configured to determine the perceptual load of the visual and dynamic scene without any measures of driver behavior while they drive, the predetermined load model being trained on pairwise comparison with an experienced test driver selecting which reference video from a pair of reference videos is more demanding on driver attention and assigning the selected reference video a higher load value, the control device being configured to:
receive a sensor output of a sensor, the sensor sensing the visual driving scene,
extract a set of scene features from the sensor output, the set of scene features representing static and dynamic information of the visual driving scene,
determine the perceptual load of the set of extracted scene features based on the predetermined load model,
the load model being predetermined based on reference video scenes each being labelled with a load value,
map the perceptual load to the sensed driving scene, and
determine a spatial and temporal intensity distribution of the perceptual load across the sensed driving scene.