US 12,277,635 B1
User verification of a generative response to a multimodal query
Harshit Kharbanda, Pleasanton, CA (US); Louis Wang, San Francisco, CA (US); Christopher James Kelley, Orinda, CA (US); and Jessica Lee, Brooklyn, NY (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Dec. 7, 2023, as Appl. No. 18/532,470.
Int. Cl. G06T 11/60 (2006.01); G06T 13/80 (2011.01)
CPC G06T 11/60 (2013.01) [G06T 13/80 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, the method comprising:
receiving, by a computing system comprising one or more processors, image data from a user device;
receiving, by the computing system, a prompt query associated with the image data;
determining, using a computer vision model, a first object from the image data that is associated with the prompt query;
receiving, by the computing system from the user device, a user indication that the first object is incorrectly labeled;
reclassifying, by the computing system, the first object based on the user indication; and
in response to receiving the user indication, generating a response using a large language model based on the reclassification of the first object.