US 12,334,075 B2
Hardware efficient automatic speech recognition
Adam Joseph Sypniewski, Dexter, MI (US); Joshua Gevirtz, Indianapolis, IN (US); Nikola Lazar Whallon, Seattle, WA (US); Anthony John Deschamps, Windsor (CA); and Scott Ivan Stephenson, Burlingame, CA (US)
Assigned to Deepgram, Inc., San Francisco, CA (US)
Filed by Deepgram, Inc., San Francisco, CA (US)
Filed on Oct. 14, 2022, as Appl. No. 17/965,960.
Prior Publication US 2024/0127819 A1, Apr. 18, 2024
Int. Cl. G10L 15/26 (2006.01); G10L 15/30 (2013.01)
CPC G10L 15/26 (2013.01) [G10L 15/30 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
receiving a plurality of requests for audio transcription;
determining a compute graph for each request, wherein the compute graph comprises one or more artificial intelligence models;
batching the requests, based on the artificial intelligence models in the compute graph of each request;
loading one or more artificial intelligence models corresponding to a batch to a hardware module;
pushing processing of a request in the batch to the hardware module, loaded with the one or more artificial intelligence models, corresponding to the batch; and
offloading, from the hardware module, artificial intelligence models not needed for processing the requests in the batch.