US 12,111,747 B1
Dynamic input-sensitive validation of machine learning model outputs and methods and systems of the same
Payal Jain, London (GB); Tariq Husayn Maonah, London (GB); Mariusz Saternus, Cracow (PL); Daniel Lewandowski, Cracow (PL); Biraj Krushna Rath, London (GB); Stuart Murray, London (GB); and Philip Davies, London (GB)
Assigned to CITIBANK, N.A., New York, NY (US)
Filed by Citibank, N.A., New York, NY (US)
Filed on May 10, 2024, as Appl. No. 18/661,532.
Application 18/661,532 is a continuation in part of application No. 18/661,519, filed on May 10, 2024.
Application 18/661,532 is a continuation in part of application No. 18/633,293, filed on Apr. 11, 2024.
Int. Cl. G06F 11/36 (2006.01); G06F 8/41 (2018.01)
CPC G06F 11/3608 (2013.01) [G06F 8/41 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory computer-readable storage medium comprising instructions thereon, wherein the instructions when executed by at least one data processor of a system, cause the system to:
receive, from a user device, an output generation request including an input for generation of an output using a first large-language model (LLM);
provide the input to the first LLM to generate the output;
determine that the output includes a first code sample for a software routine;
in response to determining that the output includes the first code sample, provide the input, an indication of the first LLM, and the first code sample to a parameter generation model to generate validation test parameters,
wherein the validation test parameters include (1) compilation instructions, (2) a virtual machine configuration, and (3) validation criteria;
configure, based on the virtual machine configuration, a virtual machine environment of the system;
compile, within the virtual machine environment and using the compilation instructions, the first code sample to generate a set of executable instructions for the software routine;
execute, within the virtual machine environment, the set of executable instructions for the software routine to generate a test output;
determine a validation indicator specifying whether the test output satisfies the validation criteria;
in response to determining that the test output satisfies the validation criteria, transmit the output to a server system enabling access to the output by the user device; and
in response to determining that the test output does not satisfy the validation criteria:
based on the validation indicator, generate a modified output including a second code sample different from the first code sample; and
transmit the modified output to the server system to enable access to the modified output by the user device.