US 12,186,026 B2
Augmented reality ready optical tracking system
Gaëtan Marti, Le Mont-sur-Lausanne (CH); and Maurice Hälg, Savigny (CH)
Assigned to Smith & Nephew, Inc., Memphis, TN (US); Smith & Nephew Orthopaedics AG, Zug (CH); and Smith & Nephew Asia Pacific Pte. Limited, Singapore (SG)
Appl. No. 17/431,384
Filed by Smith & Nephew, Inc., Memphis, TN (US); Smith & Nephew Asia Pacific Pte. Limited, Singapore (SG); and Smith & Nephew Orthopaedics AG, Zug (CH)
PCT Filed Feb. 8, 2021, PCT No. PCT/US2021/017054
§ 371(c)(1), (2) Date Aug. 16, 2021,
PCT Pub. No. WO2021/159061, PCT Pub. Date Aug. 12, 2021.
Claims priority of provisional application 63/014,803, filed on Apr. 24, 2020.
Claims priority of provisional application 62/971,448, filed on Feb. 7, 2020.
Prior Publication US 2022/0087749 A1, Mar. 24, 2022
Int. Cl. A61B 34/20 (2016.01); A61B 34/10 (2016.01); A61B 34/32 (2016.01); A61B 90/00 (2016.01); A61B 90/30 (2016.01); A61B 90/50 (2016.01); G01B 11/22 (2006.01); G01B 11/25 (2006.01); G06T 7/11 (2017.01); G06T 7/20 (2017.01); G06T 7/246 (2017.01); G06T 7/73 (2017.01); G06T 7/90 (2017.01); G06T 15/00 (2011.01); G06T 19/00 (2011.01); G16H 30/20 (2018.01); A61B 34/00 (2016.01)
CPC A61B 34/20 (2016.02) [A61B 34/10 (2016.02); A61B 34/32 (2016.02); A61B 90/30 (2016.02); A61B 90/36 (2016.02); A61B 90/39 (2016.02); A61B 90/50 (2016.02); G01B 11/22 (2013.01); G01B 11/2545 (2013.01); G06T 7/11 (2017.01); G06T 7/20 (2013.01); G06T 7/246 (2017.01); G06T 7/73 (2017.01); G06T 7/74 (2017.01); G06T 7/90 (2017.01); G06T 15/00 (2013.01); G06T 19/006 (2013.01); G16H 30/20 (2018.01); A61B 2034/105 (2016.02); A61B 2034/107 (2016.02); A61B 2034/108 (2016.02); A61B 2034/2057 (2016.02); A61B 2034/2065 (2016.02); A61B 2034/2068 (2016.02); A61B 2034/256 (2016.02); A61B 2090/309 (2016.02); A61B 2090/363 (2016.02); A61B 2090/365 (2016.02); A61B 2090/367 (2016.02); A61B 2090/3937 (2016.02); A61B 2090/397 (2016.02); A61B 2090/3979 (2016.02); A61B 2090/3995 (2016.02); A61B 2090/502 (2016.02); G06T 2207/10024 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/30196 (2013.01); G06T 2207/30204 (2013.01)] 11 Claims
OG exemplary drawing
 
1. A method for improved optical tracking, the method comprising:
capturing with a same image sensor a color image and a tracking image of an environment comprising one or more fiducials, wherein the image sensor is a red-green-blue (RGB) complementary metal-oxide semiconductor (CMOS) image sensor, wherein the tracking image is obtained during an emission of near-infrared (NIR) light within the environment, wherein a first integration time for the tracking image is shorter than a second integration time for the color image, and wherein the image sensor is unfiltered with respect to the NIR light;
processing the tracking image to determine tracking information comprising three-dimensional positional data of each of the one or more fiducials within the environment, wherein one or more of the fiducials comprise active or passive fiducial markers configured to emit or reflect the NIR light, respectively;
registering the positional data to the color image based on the tracking information, wherein the tracking information and the color image are in a same referential space; and
outputting to a display device a representation of the environment generated based on the registration and comprising contents of the color image overlaid with the registered data.