US 12,245,845 B2
System and method for heart rate measurement using facial video
Min Wu, Clarksville, MD (US); Chau-Wai Wong, Apex, NC (US); Mingliang Chen, College Park, MD (US); Qiang Zhu, College Park, MD (US); and Zachary Lazri, Gaithersburg, MD (US)
Assigned to UNIVERSITY OF MARYLAND, COLLEGE PARK, College Park, MD (US); and NORTH CAROLINA STATE UNIVERSITY, Raleigh, NC (US)
Filed by UNIVERSITY OF MARYLAND, COLLEGE PARK, College Park, MD (US); and NORTH CAROLINA STATE UNIVERSITY, Raleigh, NC (US)
Filed on Aug. 26, 2021, as Appl. No. 17/458,170.
Application 17/458,170 is a continuation in part of application No. 16/647,233, granted, now 11,501,430, previously published as PCT/US2018/051342, filed on Sep. 17, 2018.
Claims priority of provisional application 62/559,182, filed on Sep. 15, 2017.
Claims priority of provisional application 63/070,771, filed on Aug. 26, 2020.
Prior Publication US 2021/0386307 A1, Dec. 16, 2021
Int. Cl. G06K 9/00 (2022.01); A61B 5/00 (2006.01); A61B 5/024 (2006.01); A61K 35/12 (2015.01); G06N 3/02 (2006.01); G06T 3/00 (2006.01); G06T 7/20 (2017.01); G06T 7/90 (2017.01); G16H 30/40 (2018.01)
CPC A61B 5/02416 (2013.01) [A61B 5/7207 (2013.01); A61B 5/725 (2013.01); A61B 5/7264 (2013.01); G06N 3/02 (2013.01); G06T 3/00 (2013.01); G06T 7/20 (2013.01); G06T 7/90 (2017.01); G16H 30/40 (2018.01); A61B 2576/02 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/20076 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30088 (2013.01); G06T 2207/30201 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method of remote photoplethysmography (rPPG), comprising:
receiving one or more videos that include visual frames of at least one subject performing physical activity;
detecting at least one skin pixel from one or more of the visual frames of the at least one subject;
processing three color channels in red, green and blue inside the detected at least one skin pixel to produce a pulse signal with a high pulse signal-to-noise ratio (SNR);
filtering, by a normalized least mean square filter, the pulse signal to produce a first level filtered pulse signal that is free of motion artifacts;
using a frequency trace tracking method to track a pulse rate of the at least one subject based on the first level filtered pulse signal;
reconstructing the pulse signal in a time domain to produce a second level filtered pulse signal; and
estimating at least one of pulse rate or pulse rate variability of the at least one subject based at least on the second level filtered pulse signal;
wherein the restructuring employes a filter bank comprises a spectrum modification layer to suppress noise that is outside a frequency range of the pulse signal.