US 12,333,338 B2
Asynchronous multi-tenant model inferencing on streaming databases
Saigopal Thota, Fremont, CA (US); Mridul Jain, Cupertino, CA (US); Navinder Pal Singh Brar, Abohar (IN); Pragya Jain, Indore (IN); Giridhar Addepalli, Bangalore (IN); Gajendra Alias Nishad Kamat, Los Altos, CA (US); and Santos Kumar Das, Bangalore (IN)
Assigned to WALMART APOLLO, LLC, Bentonville, AR (US)
Filed by Walmart Apollo, LLC, Bentonville, AR (US)
Filed on Oct. 22, 2021, as Appl. No. 17/508,857.
Prior Publication US 2023/0128987 A1, Apr. 27, 2023
Int. Cl. G06F 9/48 (2006.01); G06F 9/50 (2006.01)
CPC G06F 9/5016 (2013.01) [G06F 9/4881 (2013.01); G06F 2209/485 (2013.01); G06F 2209/5018 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:
ingesting streaming events for processing by multiple models, wherein each of the multiple models performs a respective machine-learning inferencing;
mapping each of the streaming events to a model of the multiple models; and
storing each of the streaming events in a respective queue in a respective sequence store of multiple sequence stores, such that a respective one of the multiple models retrieves (i) a respective one of the streaming events in the respective sequence store associated with the respective one of the multiple models and (ii) a respective key corresponding to the respective one of the streaming events from a leaf store, to asynchronously perform the respective machine-learning inferencing based on content of the respective one of the streaming events, wherein the multiple models run independently and in parallel on multi-tenant threads.