US 12,076,005 B2
Video-based analysis of stapling events during a surgical procedure using machine learning
Darrick Tyler Sturgeon, Oakland, CA (US); Jocelyn Elaine Barker, San Jose, CA (US); Varun Kejriwal Goel, Oakland, CA (US); and Taylor W. Aronhalt, Loveland, OH (US)
Assigned to Verb Surgical Inc., Santa Clara, CA (US)
Filed by Verb Surgical Inc., Santa Clara, CA (US)
Filed on Mar. 22, 2023, as Appl. No. 18/188,163.
Claims priority of provisional application 63/323,046, filed on Mar. 23, 2022.
Prior Publication US 2023/0301648 A1, Sep. 28, 2023
Int. Cl. A61B 17/068 (2006.01); A61B 17/072 (2006.01); A61B 17/115 (2006.01); G06V 10/774 (2022.01); A61B 1/00 (2006.01); A61B 17/00 (2006.01); A61B 34/00 (2016.01); A61B 34/20 (2016.01); A61H 31/00 (2006.01); G06F 3/0482 (2013.01); G06V 10/764 (2022.01); G06V 10/778 (2022.01)
CPC A61B 17/068 (2013.01) [A61B 17/072 (2013.01); A61B 17/115 (2013.01); G06V 10/774 (2022.01); A61B 1/00193 (2013.01); A61B 2017/00367 (2013.01); A61B 34/20 (2016.02); A61B 34/71 (2016.02); A61H 31/00 (2013.01); G06F 3/0482 (2013.01); G06V 10/764 (2022.01); G06V 10/7788 (2022.01)] 20 Claims
OG exemplary drawing
 
11. A non-transitory computer-readable storage medium storing instructions for automatically characterizing stapling events in a surgical procedure based on an input video, the instructions when executed by a processor causing the processor to perform steps comprising:
receiving the input video of the surgical procedure;
obtaining a first machine learning model trained by a supervised learning process using annotated training videos that are labeled to indicate timing of occurrences of stapling events and one or more characteristics of each of the stapling events;
applying the first machine learning model to the input video to generate an inference that specifies a detected sequence of the stapling events according to their relative timing during the surgical procedure and one or more characteristics associated with each of the detected sequence of stapling events; and
outputting the inference of the detected sequence of stapling events.