US 11,954,905 B2
Landmark temporal smoothing
Yang Cheng, Mountain View, CA (US); Xiaoyu Xiang, Palo Alto, CA (US); Shaoyuan Xu, Palo Alto, CA (US); Qian Lin, Palo Alto, CA (US); and Jan Philip Allebach, West Lafayette, IN (US)
Assigned to Hewlett-Packard Development Company, L.P., Spring, TX (US); and Purdue Research Foundation, West Lafayette, IN (US)
Appl. No. 17/434,527
Filed by Hewlett-Packard Development Company, L.P., Spring, TX (US); and Purdue Research Foundation, West Lafayette, IN (US)
PCT Filed Jun. 28, 2019, PCT No. PCT/US2019/039907
§ 371(c)(1), (2) Date Aug. 27, 2021,
PCT Pub. No. WO2020/263277, PCT Pub. Date Dec. 30, 2020.
Prior Publication US 2022/0139107 A1, May 5, 2022
Int. Cl. G06V 10/82 (2022.01); G06T 7/73 (2017.01); G06V 10/764 (2022.01); G06V 40/16 (2022.01)
CPC G06V 10/82 (2022.01) [G06T 7/74 (2017.01); G06V 10/764 (2022.01); G06V 40/165 (2022.01); G06V 40/171 (2022.01); G06V 40/174 (2022.01); G06V 40/176 (2022.01); G06T 2207/20224 (2013.01); G06T 2207/30201 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A system comprising:
a landmark detection engine to detect landmark positions of landmarks in images based on facial detection, wherein the detected landmark positions include a first detected landmark position of a landmark in a first image and a second detected landmark position of the landmark in a second image following the first image;
an optical flow landmark engine to determine, as optical flow landmark positions, the landmark positions in the images based on optical flow of the landmarks between the images, wherein the optical flow landmark positions include:
a first optical flow landmark position of the landmark in the first image, wherein the first optical flow is determined based on optical flow of the landmark from the first detected landmark position to a second optical flow position in the second image and optical flow from the second optical flow position in the second image back to the first optical flow landmark position in the first image; and
a second optical flow landmark position of the landmark in the second image;
a landmark difference engine to determine:
a first distance between the first detected landmark position of the landmark in the first image and the first optical flow landmark position of the landmark in the first image; and
a second distance between the first detected landmark position of the landmark in the first image and the second optical flow landmark position of the landmark in the second image; and
a weighted landmark determination engine to determine, for the first image and the second image following the first image, a position for the landmark in the second image based on: the second detected landmark position; the second optical flow position of the landmark in the second image; the first distance; and the second distance.