US 12,266,040 B2
Rendering tool information as graphic overlays on displayed images of tools
Brandon D. Itkowitz, San Jose, CA (US); Simon P. DiMaio, San Carlos, CA (US); Daniel J. Halabe, Los Altos, CA (US); Christopher J. Hasser, Los Altos, CA (US); Brian David Hoffman, Mountain View, CA (US); David Q. Larkin, Menlo Park, CA (US); Catherine J. Mohr, Mountain View, CA (US); Paul W. Mohr, Mountain View, CA (US); Tao Zhao, Sunnyvale, CA (US); and Wenyi Zhao, Weston, FL (US)
Assigned to INTUITIVE SURGICAL OPERATIONS, INC., Sunnyvale, CA (US)
Filed by INTUITIVE SURGICAL OPERATIONS, INC., Sunnyvale, CA (US)
Filed on Feb. 8, 2024, as Appl. No. 18/437,072.
Application 18/437,072 is a continuation of application No. 17/213,924, filed on Mar. 26, 2021, granted, now 11,941,734.
Application 17/213,924 is a continuation of application No. 16/351,444, filed on Mar. 12, 2019, granted, now 10,984,567, issued on Apr. 20, 2021.
Application 16/351,444 is a continuation of application No. 15/946,408, filed on Apr. 5, 2018, granted, now 10,282,881, issued on May 7, 2019.
Application 15/946,408 is a continuation of application No. 13/768,047, filed on Feb. 15, 2013, granted, now 10,008,017, issued on Jun. 26, 2018.
Application 13/768,047 is a continuation in part of application No. 12/415,354, filed on Mar. 31, 2009, granted, now 9,789,608, issued on Oct. 17, 2017.
Prior Publication US 2024/0296608 A1, Sep. 5, 2024
Int. Cl. G06T 11/60 (2006.01); A61B 34/00 (2016.01); A61B 34/20 (2016.01); A61B 34/30 (2016.01); A61B 34/37 (2016.01); A61B 90/00 (2016.01); B25J 9/16 (2006.01); G05B 19/42 (2006.01); G06T 11/40 (2006.01)
CPC G06T 11/60 (2013.01) [A61B 34/30 (2016.02); A61B 34/37 (2016.02); A61B 90/36 (2016.02); B25J 9/1666 (2013.01); B25J 9/1671 (2013.01); B25J 9/1692 (2013.01); G05B 19/4202 (2013.01); G06T 11/40 (2013.01); A61B 34/20 (2016.02); A61B 2034/2061 (2016.02); A61B 34/25 (2016.02); A61B 90/361 (2016.02); A61B 2090/371 (2016.02); G05B 2219/36432 (2013.01); G05B 2219/39083 (2013.01); G05B 2219/39096 (2013.01); G05B 2219/39449 (2013.01); G05B 2219/40607 (2013.01); G05B 2219/45117 (2013.01); G05B 2219/45123 (2013.01); Y02P 90/02 (2015.11)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
a first robotic arm operatively coupleable to a first tool, the first tool comprising a working end;
a first input device communicatively coupled to the first robotic arm, the first input device being manipulatable by a first operator; and
a processor configured to:
cause an image of a work site, captured by an image capture device from a perspective of an image reference frame, to be displayed on a display, the image of the work site including an image of the working end of the first tool;
determine a position of the working end of the first tool in the image of the work site; and
render a first tool information overlay at the position of the working end of the first tool in the image of the work site, wherein the first tool information overlay visually indicates an identity of the first input device.