US 11,941,503 B2
Performing inference using an adaptive, hybrid local/remote technique
Fan Zhang, Bellevue, WA (US)
Assigned to EchoNous, Inc., Redmond, WA (US)
Filed by EchoNous, Inc., Redmond, WA (US)
Filed on Nov. 11, 2020, as Appl. No. 17/095,480.
Prior Publication US 2022/0147880 A1, May 12, 2022
Int. Cl. G06K 9/00 (2022.01); G06F 18/20 (2023.01); G06N 5/04 (2023.01); G06N 20/20 (2019.01); G06T 7/00 (2017.01); G06V 30/24 (2022.01)
CPC G06N 20/20 (2019.01) [G06F 18/285 (2023.01); G06N 5/04 (2013.01); G06T 7/0012 (2013.01); G06V 30/248 (2022.01); G06T 2207/10021 (2013.01); G06T 2207/10132 (2013.01); G06V 30/2528 (2022.01)] 30 Claims
OG exemplary drawing
 
1. A system, comprising:
an ultrasound sensing device; and
a computing device, the computing device comprising:
a communication interface configured to directly receive ultrasound echo data sensed by the ultrasound sensing device from a person, the received ultrasound echo data comprising a sequence of ultrasound images;
a memory configured to:
store a local machine learning model for producing inferences each in response to an ultrasound image in the sequence;
store a queue holding inference results each corresponding to a different ultrasound image in the sequence;
one or more processors configured to:
for each ultrasound image of the sequence, in response to its receipt by the communications interface:
attribute to the ultrasound image an identifier;
dispatch the ultrasound image to a server;
subject the ultrasound image to the local machine learning model to obtain a local inference result for the ultrasound image;
add to the queue the obtained local inference result with the ultrasound image's identifier;
aggregate the inference results contained by the queue to obtain an output inference result;
remove the oldest inference result from the queue; and
augment the ultrasound image with the visual information derived from the output inference result;
receive from the server a plurality of cloud inference results each obtained by applying a server machine learning model to one of the ultrasound images dispatched to the server, each cloud inference result being accompanied by the identifier of the ultrasound image from which it was obtained;
for each received cloud inference result, in response to its receipt, substitute the cloud inference result in the queue for the local inference result having the same identifier;
a display device configured to:
cause the augmented ultrasound images to be displayed.