US 12,081,617 B2
Networked system for real-time computer-aided augmentation of live input video stream
Raj Nair, Lexington, MA (US); Prabhudev Navali, Westford, MA (US); Sudhir Halbhavi, Bangalore (IN); and Chin-Cheng Wu, Carlisle, MA (US)
Assigned to Avesha, Inc., Chelmsford, MA (US)
Filed by Avesha, Inc., Bedford, MA (US)
Filed on Aug. 4, 2021, as Appl. No. 17/394,102.
Claims priority of provisional application 63/183,244, filed on May 3, 2021.
Claims priority of provisional application 63/061,364, filed on Aug. 5, 2020.
Prior Publication US 2022/0046292 A1, Feb. 10, 2022
Int. Cl. H04L 67/1001 (2022.01); G06N 20/00 (2019.01); H04L 41/0816 (2022.01); H04L 47/20 (2022.01); H04L 67/02 (2022.01); H04L 67/60 (2022.01); H04N 21/2187 (2011.01); H04N 21/422 (2011.01); H04N 21/431 (2011.01); H04N 21/433 (2011.01); H04N 21/466 (2011.01); H04N 21/472 (2011.01)
CPC H04L 67/1001 (2022.05) [G06N 20/00 (2019.01); H04L 41/0816 (2013.01); H04L 47/20 (2013.01); H04L 67/02 (2013.01); H04L 67/60 (2022.05); H04N 21/2187 (2013.01); H04N 21/42203 (2013.01); H04N 21/4312 (2013.01); H04N 21/4334 (2013.01); H04N 21/466 (2013.01); H04N 21/47217 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A networked system for real-time computer-aided augmentation of a live input video stream, comprising:
an endpoint system being a first networked subsystem configured and operative to handle the live input video stream and a live augmented video stream and to provide service access to a video augmentation service for the computer-aided augmentation of the live input video stream;
a backend system being a second networked subsystem providing service orchestration to orchestrate use and operation of the video augmentation service; and
an edge inference system being a third networked subsystem coupled to the endpoint system by an edge network and to the backend system via a backend network, the edge inference system being configured and co-operative with the endpoint system to provide the video augmentation service on the live input video stream and thereby generate the live augmented video stream, based on the service orchestration by the backend system,
wherein the video augmentation service is a medical imaging service for automated detection of medically relevant objects in the live input video stream and overlaying object identification features identifying the relevant objects in the live augmented video stream,
and wherein the networked system is operative to provide the video augmentation service in both a remote-live mode and in a local-live mode, the remote-live mode including (a) the edge inference system generating the live augmented video stream and providing it to the endpoint system via the edge network, and (b) the endpoint system receiving the live augmented video stream from the edge inference system via the edge network and making local use of the live augmented video stream within the endpoint system, the local-live mode including (c) the edge inference system generating augmentation metadata and providing it to the endpoint system, and (d) the endpoint system receiving the augmentation metadata from the edge inference system and adding the corresponding features to the live input video stream to generate the live augmented video stream.