US 12,239,381 B2
Augmented reality in arthroplasty surgery
Constantinos Nikou, Monroeville, PA (US); Branislav Jaramaz, Pittsburgh, PA (US); Cedric Corpa De La Fuente, Gibsonia, PA (US); and Daniel Farley, Memphis, TN (US)
Assigned to SMITH & NEPHEW, INC., Memphis, TN (US); SMITH & NEPHEW ORTHOPAEDICS AG, Zug (CH); and SMITH & NEPHEW ASIA PACIFIC PTE. LIMITED, Singapore (SG)
Appl. No. 17/426,925
Filed by Smith & Nephew, Inc., Memphis, TN (US); Smith & Nephew Asia Pacific Pte. Limited, Singapore (SG); and Smith & Nephew Orthopaedics AG, Zug (CH)
PCT Filed Feb. 4, 2020, PCT No. PCT/US2020/016561
§ 371(c)(1), (2) Date Jul. 29, 2021,
PCT Pub. No. WO2020/163316, PCT Pub. Date Aug. 13, 2020.
Claims priority of provisional application 62/939,946, filed on Nov. 25, 2019.
Claims priority of provisional application 62/885,673, filed on Aug. 12, 2019.
Claims priority of provisional application 62/864,663, filed on Jun. 21, 2019.
Claims priority of provisional application 62/801,245, filed on Feb. 5, 2019.
Claims priority of provisional application 62/801,257, filed on Feb. 5, 2019.
Prior Publication US 2022/0117669 A1, Apr. 21, 2022
Int. Cl. A61B 34/00 (2016.01); A61B 17/17 (2006.01); A61B 34/10 (2016.01); A61B 90/00 (2016.01); A61F 2/46 (2006.01); G02B 27/01 (2006.01); G06N 3/08 (2023.01); G06N 5/046 (2023.01); G06N 20/00 (2019.01); G06N 20/10 (2019.01); G09B 5/02 (2006.01); G09B 19/00 (2006.01); G09B 19/24 (2006.01); G16H 10/60 (2018.01); G16H 20/40 (2018.01); G16H 30/40 (2018.01); G16H 40/63 (2018.01); G16H 50/50 (2018.01); G16H 50/70 (2018.01); A61B 17/00 (2006.01); A61B 17/15 (2006.01); A61B 34/20 (2016.01); A61B 34/30 (2016.01); A61B 90/50 (2016.01); A61B 90/96 (2016.01); A61F 2/30 (2006.01)
CPC A61B 34/10 (2016.02) [A61B 17/1764 (2013.01); A61B 34/25 (2016.02); A61B 90/361 (2016.02); A61B 90/37 (2016.02); A61B 90/39 (2016.02); A61F 2/461 (2013.01); G02B 27/0172 (2013.01); G06N 3/08 (2013.01); G06N 5/046 (2013.01); G06N 20/00 (2019.01); G06N 20/10 (2019.01); G09B 5/02 (2013.01); G09B 19/003 (2013.01); G09B 19/24 (2013.01); G16H 10/60 (2018.01); G16H 20/40 (2018.01); G16H 30/40 (2018.01); G16H 40/63 (2018.01); G16H 50/50 (2018.01); G16H 50/70 (2018.01); A61B 2017/00199 (2013.01); A61B 2017/00526 (2013.01); A61B 17/15 (2013.01); A61B 17/17 (2013.01); A61B 2034/102 (2016.02); A61B 2034/104 (2016.02); A61B 2034/105 (2016.02); A61B 2034/107 (2016.02); A61B 2034/108 (2016.02); A61B 34/20 (2016.02); A61B 2034/2048 (2016.02); A61B 2034/2057 (2016.02); A61B 2034/2063 (2016.02); A61B 2034/2065 (2016.02); A61B 2034/2068 (2016.02); A61B 2034/2072 (2016.02); A61B 2034/2074 (2016.02); A61B 2034/252 (2016.02); A61B 2034/254 (2016.02); A61B 2034/256 (2016.02); A61B 2034/258 (2016.02); A61B 34/30 (2016.02); A61B 2090/363 (2016.02); A61B 2090/365 (2016.02); A61B 2090/371 (2016.02); A61B 2090/376 (2016.02); A61B 2090/3916 (2016.02); A61B 2090/502 (2016.02); A61B 90/96 (2016.02); A61F 2002/30952 (2013.01); A61F 2002/4633 (2013.01); G02B 2027/0138 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A surgical system comprising:
a first plurality of fiducial markers configured to be affixed to one or more patient bones;
a plurality of cameras configured to capture images of a surgical scene including the first plurality of fiducial markers, the plurality of cameras including both headset cameras and fixed cameras;
a plurality of headsets, each comprising an augmented reality display and at least one of the plurality of cameras, configured to be worn by surgical staff; and
one or more processors configured to process the images of the surgical scene and to maintain a three-dimensional patient model of at least one anatomical feature of a patient relative to a reference frame defined by the first plurality of fiducial markers, using images captured from at least a subset of the headset cameras and a subset of the fixed cameras,
wherein each of the plurality of headsets is configured to overlay contextual information about the at least one anatomical feature via the augmented reality display at a location corresponding to the at least one anatomical feature, based on the three-dimensional patient model, and wherein the one or more processors maintains a headset model of a field of view of plurality of headsets relative to the three-dimensional patient model, wherein the contextual information includes visual feedback of compliance with a surgical plan.