US 12,405,846 B2
Providing application programming interface endpoints for machine learning models
David Lisuk, New York, NY (US); and Simon Slowik, London (GB)
Assigned to Palantir Technologies Inc., Denver, CO (US)
Filed by Palantir Technologies Inc., Denver, CO (US)
Filed on Jun. 13, 2024, as Appl. No. 18/742,927.
Application 18/742,927 is a continuation of application No. 18/139,663, filed on Apr. 26, 2023, granted, now 12,045,678.
Application 18/139,663 is a continuation of application No. 17/680,859, filed on Feb. 25, 2022, granted, now 11,669,377, issued on Jun. 6, 2023.
Application 17/680,859 is a continuation of application No. 16/990,233, filed on Aug. 11, 2020, granted, now 11,288,110, issued on Mar. 29, 2022.
Claims priority of provisional application 62/889,942, filed on Aug. 21, 2019.
Prior Publication US 2024/0411628 A1, Dec. 12, 2024
Int. Cl. G06F 9/54 (2006.01); G06F 9/455 (2018.01); G06N 20/00 (2019.01); H04L 67/133 (2022.01)
CPC G06F 9/547 (2013.01) [G06F 9/45558 (2013.01); G06N 20/00 (2019.01); H04L 67/133 (2022.05); G06F 2009/45562 (2013.01); G06F 2009/45591 (2013.01); G06F 2009/45595 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
receiving a request, from a client device, the request comprising input data for a desired machine learning model;
selecting an execution environment with a loaded instance of the desired machine learning model by:
identifying one or more active virtual machines that are associated with an application programming interface (API) endpoint;
determining a virtual machine of the one or more active virtual machines with an available execution environment that has the loaded instance of the desired machine learning model; and
selecting the available execution environment to be the execution environment; and
instructing the API endpoint to forward the input data to the selected execution environment to obtain output data that is returned to the client device.