	US 11,755,008 B2
	Using plays for human-AI decision-making in remote autonomous vehicle support
Laura Cesafsky, San Francisco, CA (US); Astha Vagadia, San Jose, CA (US); Hillary Abraham, Irvine, CA (US); Sparsh Sharma, Santa Clara, CA (US); Stefan Witwicki, San Carlos, CA (US); Najamuddin Mirza Baig, San Jose, CA (US); and Omar Bentahar, Sunnyvale, CA (US)
Assigned to Nissan North America, Inc., Franklin, TN (US); and Renault S.A.S., Boulogne-Billancourt (FR)
Filed by Nissan North America, Inc., Franklin, TN (US)
Filed on Jan. 31, 2020, as Appl. No. 16/779,393.
Prior Publication US 2021/0240191 A1, Aug. 5, 2021
Int. Cl. G05D 1/00 (2006.01)

CPC G05D 1/0016 (2013.01) [G05D 1/0038 (2013.01); G05D 1/0088 (2013.01); G05D 2201/0212 (2013.01)]

20 Claims

1. An apparatus for traversing a vehicle transportation network by an autonomous vehicle (AV), comprising:

a human-autonomy teaming manager that is remote from the vehicle and comprises a first processor configured to:

select a predefined dynamic task sequence that is responsive to a condition experienced by the AV while traversing from a starting location to an ending location within the vehicle transportation network, wherein the condition results in a request for assistance from the AV, the condition is associated with a use case class of a plurality of use case classes, and the predefined dynamic task sequence is one of a plurality of predefined dynamic task sequences that delegates its tasks to multiple agents to resolve the condition;

initiate a dialogue of the predefined dynamic task sequence that prompts a human agent of the multiple agents to confirm the predefined dynamic task sequence as a selected predefined dynamic task sequence or select an other predefined dynamic task sequence grouped in the use case class;

provide, to an interface accessible to a human agent of the multiple agents, access to a library through a dialogue window, wherein the library stores available predefined dynamic task sequences including the plurality of predefined dynamic task sequences, and each predefined dynamic task sequence of the available predefined dynamic task sequences is for use with a respective use case class of the plurality of use case classes;

receive, from the interface accessible to the human agent, an input responsive to the dialogue, wherein the input confirms the predefined dynamic task sequence as the selected predefined dynamic task sequence or selects the other predefined dynamic task sequence as the selected predefined dynamic task sequence; and

manage the selected predefined dynamic task sequence to resolve the condition.

11. An apparatus for traversing a vehicle transportation network by an autonomous vehicle (AV), comprising:

an autonomous mobility agent remote from the AV and comprising a first processor configured to:

receive a request for remote assistance support from the AV responsive to a condition observed by the AV while traversing from a starting location to an ending location within the vehicle transportation network; and

a human-autonomy teaming manager remote from the AV, wherein the first processor is configured to, responsive to the request, call the human-autonomy teaming manager to select a predefined dynamic task sequence, and the human-autonomy teaming manager comprising a second processor configured to:

select the predefined dynamic task sequence that is one of a plurality of predefined dynamic task sequences using multiple agents to resolve the condition, the multiple agents including a human agent and the autonomous mobility agent;

present a dialogue of the predefined dynamic task sequence to the human agent at a user interface of the human-autonomy teaming manager, wherein a first task of the dialogue prompts the human agent to confirm the predefined dynamic task sequence as a selected predefined dynamic task sequence or select an other predefined dynamic task sequence of the plurality of predefined dynamic task sequences as the selected predefined dynamic task sequence;

provide, to the user interface, access to a library through a dialogue window, wherein the library stores available predefined dynamic task sequences including the plurality of predefined dynamic task sequences, and the library is stored in a memory of the human-autonomy teaming manager;

receive, from the user interface, an input responsive to the dialogue, wherein the input confirms the predefined dynamic task sequence as the selected predefined dynamic task sequence or selects the other predefined dynamic task sequence of the plurality of predefined dynamic task sequences within the library as the selected predefined dynamic task sequence; and

manage the selected predefined dynamic task sequence by delegating tasks of the selected predefined dynamic task sequence to resolve the condition, wherein the selected predefined dynamic task sequence includes at least one conditional task.