US 12,437,154 B1
Information extraction system for unstructured documents using retrieval augmentation providing source traceability and error control
Christopher Cirelli, Roswell, GA (US); and Lei Zhang, New York, NY (US)
Assigned to American International Group, Inc., New York, NY (US)
Filed by American International Group, Inc., New York, NY (US)
Filed on Jan. 24, 2025, as Appl. No. 18/831,432.
Int. Cl. G06F 40/289 (2020.01); G06F 11/34 (2006.01); G06F 16/3329 (2025.01)
CPC G06F 40/289 (2020.01) [G06F 11/3476 (2013.01); G06F 16/3329 (2019.01); G06F 2201/865 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for providing traceability in large language model responses, the method comprising:
generating, by one or more processors, a plurality of chunks from a document;
associating, by the one or more processors, (i) a document identifier for the document and a page identifier or (ii) a chunk identifier for each chunk of the plurality of chunks;
identifying, by the one or more processors, one or more relevant chunks from the plurality of chunks based on a search criterion;
transmitting, by the one or more processors to a large language model, a prompt comprising (i) a first request to extract particular information using the one or more relevant chunks, (ii) the one or more relevant chunks, and (iii) a second request for the large language model to identify used chunks of the one or more relevant chunks used to extract the particular information; and
storing the particular information from a response to the prompt from the large language model with (i) the document identifiers associated with the used chunks and the page identifiers associated with the used chunks or (ii) the chunk identifiers for the used chunks.