US 12,260,260 B1
Digital delegate computer system architecture for improved multi-agent large language model (LLM) implementations
Matthew J. Gorman, Burlington, CT (US); Vincent E. Haines, East Point, GA (US); Girish A. Modgil, Alpharetta, GA (US); and Brad E. Gawron, Porter Corners, NY (US)
Assigned to The Travelers Indemnity Company, Hartford, CT (US)
Filed by The Travelers Indemnity Company, Hartford, CT (US)
Filed on Mar. 29, 2024, as Appl. No. 18/622,721.
Int. Cl. G06F 9/50 (2006.01); G06F 9/46 (2006.01); G06F 9/48 (2006.01)
CPC G06F 9/5066 (2013.01) [G06F 9/468 (2013.01); G06F 9/4881 (2013.01); G06F 2209/485 (2013.01); G06F 2209/5017 (2013.01); G06F 2209/506 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A multi-Large Language Model (LLM), multi-agent, digital delegate computer system, comprising:
an identity server comprising at least one electronic processing device;
a non-transitory access entitlement data store device in communication with the identity server, the access entitlement data store device storing access entitlement data in relation to user identification data;
a multi-agent LLM server comprising a plurality of electronic processing devices and being in communication with the identity server; and
a non-transitory LLM data store device in communication with the multi-agent LLM server, the LLM data store device storing (i) instructions defining a primary LLM agent, (ii) instructions defining a primary LLM; (iii) data descriptive of a plurality of LLM tools, (iv) instructions defining, for each LLM tool of the plurality of LLM tools, a secondary LLM agent, and (v) operating instructions, that when executed by the plurality of electronic processing devices, result in:
receiving, by the multi-agent LLM server and from a user device, a user request comprising a prompt and an indication of an identifier of a user of the user device;
identifying, by the identity server and utilizing the identifier of the user to query the access entitlement data store, at least one access entitlement assigned to the user;
identifying, by an execution of the primary LLM agent by the multi-agent LLM server, and based on the at least one access entitlement assigned to the user, and by querying the data descriptive of the plurality of LLM tools, a subset of LLM tools from the plurality of LLM tools that the user is entitled access to;
generating, by the primary LLM and utilizing both the prompt and the identified subset of LLM tools from the plurality of LLM tools that the user is entitled access to, and after the identification of the subset of LLM tools from the plurality of LLM tools that the user is entitled access to, a multi-tier plan for responding to the user request, wherein the multi-tier plan defines a plurality of actions, with each action being assigned to one of the LLM tools from the identified subset of LLM tools;
executing, by the primary LLM agent, the multi-tier plan, by:
(i) calling a secondary LLM agent assigned to each respective one of the LLM tools from the identified subset of tools for the plurality of actions of the multi-tier plan; and
(ii) receiving, from each secondary LLM agent and in response to the calling, a response for each of the actions of the multi-tier plan;
constructing, by the primary LLM agent and utilizing the responses for the actions of the multi-tier plan, a user response; and
transmitting, by the primary LLM agent and to the user device, the user response.