US 12,443,595 B1
Method and system for verification with large language models for data querying
Udari Madhushani Sehwag, Jersey City, NJ (US); Yanchao Sun, Jersey City, NJ (US); Kassiani Papasotiriou, New York, NY (US); Jared Vann, London (GB); and Sumitra Ganesh, Short Hills, NJ (US)
Assigned to JPMORGAN CHASE BANK, N.A., New York, NY (US)
Filed by JPMorgan Chase Bank, N.A., New York, NY (US)
Filed on Jul. 3, 2024, as Appl. No. 18/763,193.
Claims priority of application No. 20240100465 (GR), filed on Jun. 26, 2024.
Int. Cl. G06F 16/242 (2019.01); G06F 11/34 (2006.01); G06F 16/2452 (2019.01); G06F 16/248 (2019.01)
CPC G06F 16/2425 (2019.01) [G06F 11/3409 (2013.01); G06F 16/24522 (2019.01); G06F 16/248 (2019.01)] 17 Claims
OG exemplary drawing
 
1. A method for verifying a structured query language (SQL) query, the method being implemented by at least one processor, the method comprising:
receiving, by the at least one processor, a first request to retrieve first data that is accessible via a database;
identifying, by the at least one processor, an intention of the first request;
generating, by the at least one processor via a first large language model (LLM), a first SQL query to retrieve the first data from the database, based on the first request;
retrieving, by the at least one processor, schema information associated with the database;
training, by the at least one processor, a second LLM model using the schema information;
computing, by the at least one processor via the second LLM, sample values for a plurality of simulations, based on the schema information and the first SQL query;
predicting, by the at least one processor via a third LLM, an output of the first SQL query, based on the sample values, the schema information, and the first SQL query;
verifying, by the at least one processor, that the first SQL query corresponds to the identified intention of the first request based on whether the predicted output matches the identified intention of the first request; and
generating, by the at least one processor and based on a result of the verifying, a second SQL query to retrieve the first data that is accessible via the database, when the predicted output of the first SQL query does not match the identified intention of the first request.