| CPC G06N 3/0442 (2023.01) [G06N 3/0475 (2023.01)] | 17 Claims |
|
1. A computer-implemented method comprising:
receiving, from a model computing environment, data characterizing a multimodal prompt for ingestion by a generative artificial intelligence (GenAI) model, the GenAI model comprising a pipeline including an encoder for processing the received data prior to ingestion by a plurality of layers including at least one intermediate layer, the GenAI model further comprising a transformer architecture with residual streams;
redirecting the multimodal prompt to a quantized proxy of the GenAI model that replicates a layer topology of the GenAI model and is more compact;
capturing an intermediate result derived from the at least one intermediate layer of the quantized proxy of the GenAI model responsive to the inputting of the received data after it is processed by the encoder, wherein the intermediate result comprises residual-stream activation vectors derived from the at least one intermediate layer of the quantized proxy of the GenAI model via forward-pass hooks applied to the transformer layers;
determining, based on the intermediate result and using a classifier, whether the prompt seeks to cause the GenAI model to behave in an undesired manner, the classifier being trained using a data set generated using a policy mapping intermediate results to undesired model behavior including a plurality of enumerated prohibited actions; and
providing data characterizing the determination to a consuming application or process, the consuming application or process:
preventing the multimodal prompt from being ingested by the GenAI model in response to determining that the prompt elicits undesired actions by the GenAI model; and
allowing the multimodal prompt to be ingested by the GenAI model in response to determining that the prompt does not elicit undesired actions by the GenAI model.
|