US 12,067,983 B2
Robust useful and general task-oriented virtual assistants
Biplav Srivastava, Chapin, SC (US); Kausik Lakkaraju, Columbia, SC (US); Revathy Venkataramanan, Columbia, SC (US); Vishal Pallagani, Columbia, SC (US); Vedant Khandelwal, Columbia, SC (US); and Hong Yung Yip, Columbia, SC (US)
Assigned to UNIVERSITY OF SOUTH CAROLINA, Columbia, SC (US)
Filed by UNIVERSITY OF SOUTH CAROLINA, Columbia, SC (US)
Filed on Apr. 6, 2022, as Appl. No. 17/714,508.
Claims priority of provisional application 63/284,272, filed on Nov. 30, 2021.
Claims priority of provisional application 63/185,168, filed on May 6, 2021.
Prior Publication US 2022/0358922 A1, Nov. 10, 2022
Int. Cl. G10L 15/22 (2006.01); G06F 3/16 (2006.01); G06F 9/38 (2018.01)
CPC G10L 15/22 (2013.01) [G06F 3/167 (2013.01); G06F 9/38 (2013.01); G10L 2015/221 (2013.01); G10L 2015/223 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
using at least one or more processors programmed to perform acts of:
accessing information specifying at least one corresponding user-specified task to be achieved;
in response to the user-specified task, causing a virtual assistant executing on the one or more processors to:
discover instructions from online data sources for achieving the user-specified task,
assist the user in performing the instructions or monitor execution of the instructions by the user,
observe failures of execution of the instructions, and
help recover from action and instruction level failures till completion of the user-specified task,
wherein the one or more processors are further programmed for causing the virtual assistant to operate with at least one of the following path workflows:
Path-1: C1→C2→C7→C2→C6→C2→C7→C2→C4 (without personalization),
Path-2: C1→C2→C7→C2→C6→C2→C3→C2→C7→C2→C4 (with personalization),
Path-3: C1→C2→C7→C2→C4,
Path-4: C1→C2→C3→C2→C7→C2→C4,
Path-5: C1→C2→C7→C2→C4 (if recovery plan exists in the table), or
Path-6: C1→C2→C7→C2→C4 (if recovery plan does not exist in the table),
where
C1 comprises a Multimodal Pipeline to process the user query, identify the problem and its domain,
C2 comprises an Orchestrator responsible for choosing the right workflow depending on the user input,
C3 comprises the VA Storage,
C4 comprises an Executor,
C6 comprises an Instruction Miner for mining relevant instructions from reliable sources for a given user query or task, and
C7 comprises a Planner responsible for generating a plan for the given task or for retrieving a plan if a plan already exists for that particular task.