US 12,376,766 B2
System and method for determining heart beat features
Andrey Viktorovich Filimonov, Kamenki (RU); Ivan Sergeevich Shishalov, Nizhniy Novgorod (RU); Roman Aleksandrovich Ershov, Nizhny Novgorod (RU); and Andrey Sergeevich Shilov, Nizhny Novgorod (RU)
Assigned to HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH, Karlsbad (DE)
Filed by Harman Becker Automotive Systems GmbH, Karlsbad (DE)
Filed on Oct. 19, 2021, as Appl. No. 17/451,470.
Claims priority of application No. PCT/RU2020/000555 (WO), filed on Oct. 19, 2020.
Prior Publication US 2022/0117517 A1, Apr. 21, 2022
Int. Cl. A61B 5/1171 (2016.01); A61B 5/00 (2006.01); A61B 5/021 (2006.01); A61B 5/024 (2006.01); A61B 5/026 (2006.01); A61B 5/18 (2006.01)
CPC A61B 5/1176 (2013.01) [A61B 5/0077 (2013.01); A61B 5/02108 (2013.01); A61B 5/02405 (2013.01); A61B 5/026 (2013.01); A61B 5/18 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A system for determining one or more heart beat features, the system comprising:
a visible light camera;
an infrared camera; and
a computing device;
wherein:
the visible light camera is operable to capture a series of visible light images of a face of a user;
the infrared camera is operable to capture, during capturing the series of the visible light images, at least one infrared image of the face; and
the computing device implemented in hardware is operable to:
detect, on the at least one infrared image, one or more first face landmarks;
detect, on at least one of the visible light images, one or more second face landmarks;
determine a first confidence value of the one or more first face landmarks;
determine a second confidence value of the one or more second face landmarks, wherein the second confidence value is expressed on the same scale as the first confidence value;
compare the first confidence value to the second confidence value;
in response to the first confidence value being greater than the second confidence value, determine a location of one or more skin patches on the series of visible light images based on the one or more first face landmarks;
in response to the second confidence value being equal to or greater than the first confidence value, determine the location of the one or more skin patches on the series of visible light images based on the one or more second face landmarks; and
determine the one or more heart beat features based on the determined location of the one or more skin patches.