US 12,314,991 B2
Method and system for controlling a graphical user interface by telephone
Kamyar Mohajer, Mississauga (CA); Keyvan Mohajer, Los Gatos, CA (US); James Hom, Palo Alto, CA (US); and Evelyn Jiang, Cupertino, CA (US)
Assigned to SoundHound AI IP, LLC.
Filed by SoundHound, Inc., Santa Clara, CA (US)
Filed on Aug. 22, 2021, as Appl. No. 17/408,476.
Prior Publication US 2023/0059765 A1, Feb. 23, 2023
Int. Cl. G06Q 30/0601 (2023.01)
CPC G06Q 30/0619 (2013.01) [G06Q 30/0623 (2013.01); G06Q 30/0631 (2013.01); G06Q 30/0633 (2013.01); G06Q 30/0643 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method for an interactive voice and GUI system, comprising:
receiving, at a voice ordering server, a voice telephone call from a mobile device associated with a telephone number;
transmitting the voice telephone call to a middleware reflecting voice from the telephone call;
performing speech recognition on audio information of the voice telephone call;
determining response data in an optimal format based on the audio information of the voice telephone call, wherein the response data comprises interactive audio data and visual data, and wherein the audio data is delivered via synthesized speech on the voice telephone call and the visual data is displayed on the mobile device via a URL;
sending, after confirming via the audio data by the voice ordering server, a text message to the mobile device associated with the telephone number, the text message containing the URL to deliver the visual data displayed on the mobile device;
receiving a request from the mobile device to the URL;
serving code that when rendered by the mobile device, provides a graphical display of a first choice associated with the URL, wherein the first choice is recognized through speech recognition by the voice ordering server;
performing speech recognition on audio received via the mobile device to interpret the interaction of the voice telephone call;
updating the graphical display based on the interaction of the voice telephone call;
serving code that, when rendered by the mobile device, provides an updated graphical display acknowledging the updated first choice; and
terminating the URL upon completion of the voice telephone call.