US 12,406,445 B2
Ghost visualization for augmented reality (AR) surgical systems and methods
Alec Paul Birkbeck, Leeds (GB); and Daniel Girardeau-Montaut, Gieres (FR)
Assigned to DePuy Synthes Products, Inc., Raynham, MA (US)
Filed by Depuy Synthes Products, Inc., Raynham, MA (US)
Filed on Apr. 17, 2023, as Appl. No. 18/135,638.
Prior Publication US 2024/0346773 A1, Oct. 17, 2024
Int. Cl. G06T 19/00 (2011.01); A61B 90/00 (2016.01); G06T 7/62 (2017.01); G06T 7/70 (2017.01); G06T 7/90 (2017.01); G06T 15/20 (2011.01)
CPC G06T 19/006 (2013.01) [A61B 90/37 (2016.02); G06T 7/62 (2017.01); G06T 7/70 (2017.01); G06T 7/90 (2017.01); G06T 15/205 (2013.01); A61B 2090/365 (2016.02); A61B 2090/371 (2016.02); G06T 2207/10024 (2013.01); G06T 2207/20104 (2013.01); G06T 2207/30004 (2013.01); G06T 2207/30196 (2013.01); G06T 2210/41 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A viewing system, comprising:
an augmented reality (AR) system having a first camera with a first point of view, the AR system having an AR display configured to display augmented reality information overlaid over a real world scene, the first point of view of the first camera having a same point of view as the AR display, and the scene containing a first physical object in a field of view of the first camera;
a second camera having a second point of view different from the first point of view, wherein the first physical object is in the field of view of the second camera; and
a controller configured to:
receive an input that the first physical object in the real world scene has become obscured by a second physical object from the first point of view of the first camera;
determine a position of the second object that has obscured the first object;
determine a perimeter of the second object;
display augmented reality information representing the determined perimeter of the second object;
extract images from the second camera of the first physical object; and
display the extracted images as augmented reality information inside the perimeter of the second object, such that the first object appears to be visible through the second object.