US 12,112,269 B2
Mixed reality-aided surgical assistance in orthopedic surgical procedures
W. Matthew Kuester, Chaska, MN (US); Damien Cariou, Loperhet (FR); and Yannick Morvan, Saint Renan (FR)
Assigned to Howmedica Osteonics Corp., Mahwah, NJ (US)
Filed by HOWMEDICA OSTEONICS CORP., Mahwah, NJ (US)
Filed on Dec. 10, 2020, as Appl. No. 17/117,708.
Application 17/117,708 is a continuation of application No. PCT/US2019/037003, filed on Jun. 13, 2019.
Claims priority of provisional application 62/687,014, filed on Jun. 19, 2018.
Claims priority of provisional application 62/739,406, filed on Oct. 1, 2018.
Claims priority of provisional application 62/778,789, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,791, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,782, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,796, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,772, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,760, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,788, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,778, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,797, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,764, filed on Dec. 12, 2018.
Claims priority of provisional application 62/778,774, filed on Dec. 12, 2018.
Claims priority of provisional application 62/804,383, filed on Feb. 12, 2019.
Claims priority of provisional application 62/804,402, filed on Feb. 12, 2019.
Claims priority of provisional application 62/804,392, filed on Feb. 12, 2019.
Prior Publication US 2021/0097886 A1, Apr. 1, 2021
Int. Cl. G06N 3/08 (2023.01); A61B 5/00 (2006.01); A61B 5/11 (2006.01); A61B 17/14 (2006.01); A61B 17/16 (2006.01); A61B 17/17 (2006.01); A61B 34/00 (2016.01); A61B 34/10 (2016.01); A61B 90/00 (2016.01); A61B 90/92 (2016.01); A61F 2/40 (2006.01); G02B 27/00 (2006.01); G02B 27/01 (2006.01); G06F 3/01 (2006.01); G06F 3/04815 (2022.01); G06F 3/0482 (2013.01); G06F 30/10 (2020.01); G06F 30/20 (2020.01); G06N 20/00 (2019.01); G06T 7/00 (2017.01); G06T 7/11 (2017.01); G06T 7/55 (2017.01); G06T 11/00 (2006.01); G06T 19/00 (2011.01); G06T 19/20 (2011.01); G09B 5/06 (2006.01); G09B 9/00 (2006.01); G09B 19/00 (2006.01); G09B 23/28 (2006.01); G16H 20/40 (2018.01); G16H 30/20 (2018.01); G16H 30/40 (2018.01); G16H 40/20 (2018.01); G16H 40/60 (2018.01); G16H 40/63 (2018.01); G16H 40/67 (2018.01); G16H 50/30 (2018.01); G16H 50/50 (2018.01); G16H 50/70 (2018.01); G16H 70/20 (2018.01); G16H 70/60 (2018.01); G16H 80/00 (2018.01); H04N 13/122 (2018.01); H04N 13/332 (2018.01); A61B 17/00 (2006.01); A61B 17/15 (2006.01); A61B 34/20 (2016.01); A61B 90/50 (2016.01); A61F 2/46 (2006.01); G06F 3/0483 (2013.01); G06N 3/04 (2023.01); G16H 50/20 (2018.01)
CPC G16H 20/40 (2018.01) [A61B 5/1114 (2013.01); A61B 5/1121 (2013.01); A61B 5/1127 (2013.01); A61B 5/681 (2013.01); A61B 17/142 (2016.11); A61B 17/1604 (2013.01); A61B 17/1626 (2013.01); A61B 17/1659 (2013.01); A61B 17/1684 (2013.01); A61B 17/1703 (2013.01); A61B 34/10 (2016.02); A61B 34/25 (2016.02); A61B 90/08 (2016.02); A61B 90/36 (2016.02); A61B 90/361 (2016.02); A61B 90/37 (2016.02); A61B 90/39 (2016.02); A61B 90/92 (2016.02); A61F 2/40 (2013.01); A61F 2/4081 (2013.01); G02B 27/0075 (2013.01); G02B 27/017 (2013.01); G02B 27/0172 (2013.01); G06F 3/011 (2013.01); G06F 3/04815 (2013.01); G06F 3/0482 (2013.01); G06F 30/10 (2020.01); G06F 30/20 (2020.01); G06N 3/08 (2013.01); G06N 20/00 (2019.01); G06T 7/0012 (2013.01); G06T 7/11 (2017.01); G06T 7/55 (2017.01); G06T 11/00 (2013.01); G06T 19/006 (2013.01); G06T 19/20 (2013.01); G09B 5/06 (2013.01); G09B 9/00 (2013.01); G09B 19/003 (2013.01); G09B 23/28 (2013.01); G16H 30/20 (2018.01); G16H 30/40 (2018.01); G16H 40/20 (2018.01); G16H 40/60 (2018.01); G16H 40/63 (2018.01); G16H 40/67 (2018.01); G16H 50/30 (2018.01); G16H 50/50 (2018.01); G16H 50/70 (2018.01); G16H 70/20 (2018.01); G16H 70/60 (2018.01); G16H 80/00 (2018.01); H04N 13/122 (2018.05); H04N 13/332 (2018.05); A61B 5/744 (2013.01); A61B 2017/00115 (2013.01); A61B 2017/00119 (2013.01); A61B 2017/00123 (2013.01); A61B 17/151 (2013.01); A61B 17/1775 (2016.11); A61B 17/1778 (2016.11); A61B 2034/102 (2016.02); A61B 2034/104 (2016.02); A61B 2034/105 (2016.02); A61B 2034/107 (2016.02); A61B 2034/108 (2016.02); A61B 2034/2048 (2016.02); A61B 2034/2051 (2016.02); A61B 2034/2055 (2016.02); A61B 2034/2065 (2016.02); A61B 2034/2068 (2016.02); A61B 2034/252 (2016.02); A61B 2034/254 (2016.02); A61B 2090/062 (2016.02); A61B 2090/067 (2016.02); A61B 2090/0801 (2016.02); A61B 2090/0807 (2016.02); A61B 2090/365 (2016.02); A61B 2090/366 (2016.02); A61B 2090/367 (2016.02); A61B 2090/368 (2016.02); A61B 2090/373 (2016.02); A61B 2090/374 (2016.02); A61B 2090/3762 (2016.02); A61B 2090/378 (2016.02); A61B 2090/3937 (2016.02); A61B 2090/3945 (2016.02); A61B 2090/397 (2016.02); A61B 2090/502 (2016.02); A61B 2505/05 (2013.01); A61B 2562/0219 (2013.01); A61F 2002/4011 (2013.01); A61F 2/4606 (2013.01); A61F 2/4612 (2013.01); A61F 2002/4633 (2013.01); A61F 2002/4658 (2013.01); A61F 2002/4668 (2013.01); G02B 2027/0141 (2013.01); G02B 2027/0174 (2013.01); G06F 3/0483 (2013.01); G06N 3/04 (2013.01); G06T 2200/24 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/20036 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30008 (2013.01); G06T 2207/30052 (2013.01); G06T 2207/30204 (2013.01); G06T 2210/41 (2013.01); G06T 2219/2004 (2013.01); G06V 2201/03 (2022.01); G16H 50/20 (2018.01)] 23 Claims
OG exemplary drawing
 
1. A system comprising:
a first device configured to display a first presentation to a first user, wherein the first device comprises a mixed-reality device and the first presentation includes one or more virtual elements configured to assist the first user in an ongoing orthopedic surgical procedure; and
a second device configured to display a second presentation to a second user, wherein the second device comprises either a mixed-reality device or a virtual reality device and the second user provides surgical assistance on the ongoing orthopedic surgical procedure,
wherein the first device is configured request assistance from the second user in response to input from the first user,
wherein, in response to the first device requesting assistance, the second device is further configured to display content that informs the second user on previously-executed steps of the ongoing orthopedic surgical procedure, wherein the content includes images or videos of the previously-executed steps of the ongoing surgical procedure that were captured during the ongoing surgical procedure, and wherein the content is indexed such that the second user can select particular surgical steps or stages.