US 12,468,712 B2
Device-agnostic framework to measure reliability during user interactions
Shashwat Lal Das, San Francisco, CA (US); Prithwish Mukherjee, Mountain View, CA (US); Xiaoyi Sheng, Sunnyvale, CA (US); Zilong Huang, Mountain View, CA (US); Prashant Gupta, Mountain View, CA (US); and Alexander Sparber, Waedenswil (CH)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on Apr. 22, 2024, as Appl. No. 18/642,318.
Prior Publication US 2025/0328534 A1, Oct. 23, 2025
Int. Cl. G06F 16/00 (2019.01); G06F 16/2457 (2019.01); G06F 16/28 (2019.01)
CPC G06F 16/24573 (2019.01) [G06F 16/285 (2019.01)] 9 Claims
OG exemplary drawing
 
1. A method implemented using one or more processors, the method comprising:
identifying metadata associated with a user query that is directed to an interactive assistant application;
processing the metadata associated with the user query to determine a classification category to which the user query belongs,
wherein processing the metadata associated with the user query includes:
determining a respective label, from a plurality of predefined labels, for each of one or more stages of the interactive assistant application handling the user query, the one or more stages belonging to a plurality of predefined stages of the interactive assistant application, and
determining the classification category based on:
the respective label for each of the one or more stages of the interactive assistant application handling the user query, and
on a total period of time from invocation of the interactive assistant application to complete rendering of a response responsive to the user query satisfying a total threshold; and
generating, based on the classification category determined for the user query, an alert to one or more entities tasked with overseeing one of the plurality of predefined stages of the interactive assistant application handling the user query in response to the one of the plurality of predefined stages being determined to have a label indicating unsatisfactory handling of the user query during the one of the plurality of predefined stages.