| CPC G06F 40/40 (2020.01) | 12 Claims |
|
1. A computer-implemented method for managing a generative artificial intelligence (AI) model, the method comprising:
receiving a question to be provided to a generative AI model;
retrieving, by a content retrieval model, a content from a knowledge base based on the question;
receiving an answer generated by the generative AI model based on the question provided to the generative AI model;
providing the question, the answer, and the content to a natural language processing (NLP) model;
generating, by the NLP model, a first similarity metric between the question and the content;
generating, by the NLP model, a second similarity metric between the answer and the content;
generating a confidence metric based on the first similarity metric and the second similarity metric, wherein the confidence metric is an indication as to whether the answer is a desirable response to the question; and
managing the generative AI model based on the confidence metric, wherein managing the generative AI model based on the confidence metric includes:
training the generative AI model using a training set, wherein:
the question is included in the training set; and
the confidence metric is used as an input in a feedback loop in the training of the generative AI model;
using the trained generative AI model to generate a second answer based on a second question provided by a user to the trained generative AI model, wherein the trained generative AI model is in use by the user; and
preventing outputting the second answer to the user based on a confidence metric indicating that the second answer is not the desirable response to the second question.
|