US 11,854,230 B2
Physical keyboard tracking
Jan Herling, Seattle, WA (US); and Daniel Scharstein, Weybridge, VT (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed on Dec. 1, 2020, as Appl. No. 17/108,617.
Prior Publication US 2022/0172397 A1, Jun. 2, 2022
Int. Cl. G06T 7/11 (2017.01); G06T 7/60 (2017.01); G06T 7/73 (2017.01); G06V 10/75 (2022.01)
CPC G06T 7/75 (2017.01) [G06T 7/11 (2017.01); G06T 7/60 (2013.01); G06V 10/751 (2022.01); G06T 2207/20021 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising, by a computing device:
capturing an image from a camera viewpoint, the image depicting a physical keyboard;
detecting one or more shape features of the physical keyboard depicted in the image by comparing pixels of the image to a predetermined shape template, wherein each of the one or more shape features of the physical keyboard corresponds to visual characteristics of a space between two or more keyboard keys of the physical keyboard, wherein comparing the pixels comprises evaluating gradients of pixel intensities of the pixels;
accessing predetermined shape features of a keyboard model associated with the physical keyboard, wherein each of the predetermined shape features of the keyboard model defines visual characteristics of a space between two or more keyboard keys of the keyboard model; and
determining a pose of the physical keyboard based on comparisons between (1) the detected one or more shape features of the physical keyboard and (2) projections of the predetermined shape features of the keyboard model toward the camera viewpoint.