US 12,436,886 B2
Caching strategy based on model execution time
Aneesh Dahiya, Zurich (CH); and Renata Khasanova, Zurich (CH)
Assigned to Oracle International Corporation, Redwood Shores, CA (US)
Filed by Oracle International Corporation, Redwood Shores, CA (US)
Filed on Jan. 4, 2024, as Appl. No. 18/403,946.
Prior Publication US 2025/0225076 A1, Jul. 10, 2025
Int. Cl. G06F 12/0802 (2016.01); G06F 16/22 (2019.01); G06N 20/00 (2019.01)
CPC G06F 12/0802 (2013.01) [G06F 16/2255 (2019.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving a first input for a model, wherein the model is a deterministic model;
generating, in accordance with the model, a first output corresponding to the first input, wherein the generating is performed in a first computation time;
storing the first input and the first output as a first input-output pair in a cache, the cache comprising a plurality of elements each including an input-output pair and a corresponding computation time associated therewith, wherein:
for each element in the plurality of elements, a corresponding computation time is an amount of time for the model to compute the output from the input in the input-output pair of the element,
the first input-output pair has a priority in the cache according to the first computation time, and
replacement of elements in the cache is based on the corresponding computation times of the elements;
subsequently receiving a second input for the model that is a duplicate of the first input; and
generating a second output corresponding to the second input by retrieving the first output from the cache.