US 12,094,205 B2
User switching detection during robotic surgeries using deep learning
Meysam Torabi, Union City, CA (US)
Assigned to Verb Surgical Inc., Santa Clara, CA (US)
Filed by Verb Surgical Inc., Santa Clara, CA (US)
Filed on Oct. 5, 2022, as Appl. No. 17/960,680.
Application 17/960,680 is a continuation of application No. 17/017,540, filed on Sep. 10, 2020, granted, now 11,488,382.
Prior Publication US 2023/0036019 A1, Feb. 2, 2023
Int. Cl. G06V 20/00 (2022.01); A61B 34/30 (2016.01); G06F 18/214 (2023.01); G06F 18/24 (2023.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G06T 3/4046 (2024.01); G06T 3/60 (2006.01); G06T 7/11 (2017.01); G06T 7/70 (2017.01); G06V 10/40 (2022.01); G06V 20/40 (2022.01)
CPC G06V 20/41 (2022.01) [A61B 34/30 (2016.02); G06F 18/214 (2023.01); G06F 18/24 (2023.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06T 3/4046 (2013.01); G06T 3/60 (2013.01); G06T 7/11 (2017.01); G06T 7/70 (2017.01); G06V 10/40 (2022.01); G06V 20/46 (2022.01); A61B 2034/301 (2016.02); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20132 (2013.01); G06T 2207/30196 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for detecting user switching events at a surgeon console in a robotic surgical system, the method comprising:
receiving a first sequence of video images capturing a user seating area of the surgeon console;
processing, using a user-presence/absence classifier, the first sequence of video images to detect a first user exiting the user seating area after detecting the first user being present at the user seating area;
receiving a second sequence of video images capturing the user seating area;
processing, using the user-presence/absence classifier, the second sequence of video images to detect a second user entering the user seating area;
generating a window on a monitor of the surgeon console prompting the second user to respond whether the second user is the same user as the first user, wherein the window is generated after a minimum time threshold of detecting the second user at the user seating area following a detecting of a user absence at the user seating area based on the processing of the second sequence of video images, thereby avoiding false positives;
receiving a response from the second user;
determining whether the second user is the same as the first user based on the received response; and
determining that a user switching event has occurred at the surgeon console in response to determining that the second user is not the same as the first user.