US 12,423,064 B2
Optimizing behavior and deployment of large language models
David Herron Auld, Seattle, WA (US); Kanio Georgiev Dimitrov, Redmond, WA (US); Jafar Mahmoud Al-Kofahi, Bothell, WA (US); Jonathan Richard Malsan, Redmond, WA (US); Diana Andrea Iftimie, Sammamish, WA (US); Natasha Kohli, Bellevue, WA (US); Chenmin Liu, Seattle, WA (US); Christopher Diego Kinney, Redmond, WA (US); Haizhen Zhang, Bothell, WA (US); Daniel Akintola Fatade, Houston, TX (US); Yousef Al-Kofahi, Niskayuna, NY (US); and Charles David Williams, Seattle, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Jun. 27, 2023, as Appl. No. 18/214,716.
Claims priority of provisional application 63/453,011, filed on Mar. 17, 2023.
Prior Publication US 2024/0311094 A1, Sep. 19, 2024
Int. Cl. G06F 8/33 (2018.01); G06F 8/35 (2018.01); G06F 8/36 (2018.01); G06F 8/60 (2018.01)
CPC G06F 8/33 (2013.01) [G06F 8/35 (2013.01); G06F 8/36 (2013.01); G06F 8/60 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method, comprising:
creating an experiment with code blocks that operate on a common data model for a large language model solution, wherein the code blocks provide instructions for an output of a pretrained large language model for the large language model solution;
creating model input using the code blocks;
providing the model input to the pretrained large language model;
receiving model output from the pretrained large language model in response to running the model input;
comparing the model output to a production threshold for the large language model solution;
creating a next experiment with modified code blocks for the large language model solution in response to determining the production threshold is not achieved with the experiment;
continuing to provide different experiments with different code blocks for the large language model solution until the production threshold is achieved; and
deploying the large language model solution into a production environment in response to a production threshold being achieved with the different experiments.