US 12,068,068 B2
Cooperative composite video streams layered onto the surgical site and instruments
Frederick E Shelton, IV, Hillsboro, OH (US)
Assigned to Cilag GmbH International, Zug (CH)
Filed by Cilag GmbH International, Zug (CH)
Filed on Jul. 23, 2021, as Appl. No. 17/384,270.
Claims priority of provisional application 63/224,813, filed on Jul. 22, 2021.
Prior Publication US 2023/0022604 A1, Jan. 26, 2023
Int. Cl. A61B 34/10 (2016.01); A61B 17/00 (2006.01); A61B 18/12 (2006.01); A61B 34/00 (2016.01); A61B 34/20 (2016.01); A61B 34/30 (2016.01); A61B 34/32 (2016.01); A61B 90/00 (2016.01); G05B 13/02 (2006.01); G06F 3/14 (2006.01); G06F 3/16 (2006.01); G06F 9/48 (2006.01); G06F 9/54 (2006.01); G06F 13/40 (2006.01); G06F 16/21 (2019.01); G06F 16/28 (2019.01); G06N 20/00 (2019.01); G06Q 10/30 (2023.01); G06T 11/60 (2006.01); G08B 5/22 (2006.01); G10L 15/22 (2006.01); G16H 10/60 (2018.01); G16H 15/00 (2018.01); G16H 20/40 (2018.01); G16H 30/40 (2018.01); G16H 40/20 (2018.01); G16H 40/40 (2018.01); G16H 40/63 (2018.01); G16H 40/67 (2018.01); G16H 50/20 (2018.01); G16H 50/70 (2018.01); H04L 1/22 (2006.01); H04L 41/12 (2022.01); H04L 65/80 (2022.01); H04L 67/12 (2022.01); H04L 67/125 (2022.01); H04N 5/272 (2006.01); H04N 7/15 (2006.01); A61B 8/06 (2006.01); A61B 18/00 (2006.01); G06F 21/62 (2013.01); G06F 40/169 (2020.01); G16H 30/20 (2018.01); H02J 7/00 (2006.01)
CPC G16H 40/20 (2018.01) [A61B 17/00 (2013.01); A61B 18/1206 (2013.01); A61B 34/10 (2016.02); A61B 34/20 (2016.02); A61B 34/25 (2016.02); A61B 34/30 (2016.02); A61B 34/32 (2016.02); A61B 90/08 (2016.02); A61B 90/37 (2016.02); G05B 13/0265 (2013.01); G06F 3/14 (2013.01); G06F 3/1423 (2013.01); G06F 3/167 (2013.01); G06F 9/4881 (2013.01); G06F 9/542 (2013.01); G06F 13/4068 (2013.01); G06F 16/211 (2019.01); G06F 16/284 (2019.01); G06F 16/285 (2019.01); G06N 20/00 (2019.01); G06Q 10/30 (2013.01); G06T 11/60 (2013.01); G08B 5/22 (2013.01); G10L 15/22 (2013.01); G16H 10/60 (2018.01); G16H 15/00 (2018.01); G16H 20/40 (2018.01); G16H 30/40 (2018.01); G16H 40/40 (2018.01); G16H 40/63 (2018.01); G16H 40/67 (2018.01); G16H 50/20 (2018.01); G16H 50/70 (2018.01); H04L 1/22 (2013.01); H04L 41/12 (2013.01); H04L 65/80 (2013.01); H04L 67/12 (2013.01); H04L 67/125 (2013.01); H04N 5/272 (2013.01); H04N 7/15 (2013.01); A61B 8/06 (2013.01); A61B 2017/00221 (2013.01); A61B 2018/00702 (2013.01); A61B 2018/00994 (2013.01); A61B 2034/2072 (2016.02); A61B 2034/254 (2016.02); A61B 2090/364 (2016.02); A61B 2090/365 (2016.02); A61B 2090/373 (2016.02); G06F 21/6245 (2013.01); G06F 40/169 (2020.01); G10L 2015/223 (2013.01); G16H 30/20 (2018.01); H02J 7/0063 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computing system comprising:
a processor configured to:
obtain a surgical video stream associated with a surgical procedure;
obtain overlay content associated with the surgical procedure;
identify a marker in a first frame of the surgical video stream;
determine a first overlay region location in the first frame of the surgical video stream for overlaying the overlay content based on the marker in the first frame of the surgical video stream;
identify the marker in a second frame of the surgical video stream;
determine a second overlay region location in the second frame of the surgical video stream for overlaying the overlay content based on the marker in the second frame of the surgical video stream; and
generate a composite video stream based on the first overlay region location and the second overlay region location.