US 12,483,522 B1
Natural language communications with an autonomous vehicle
Christopher Paul Urmson, Los Altos, CA (US); Sterling J. Anderson, Sunnyvale, CA (US); James Andrew Bagnell, Pittsburgh, PA (US); Jason Leu, Pittsburgh, PA (US); and Colin Mease, Pittsburgh, PA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Aurora Operations, Inc., Pittsburgh, PA (US)
Filed on Dec. 30, 2024, as Appl. No. 19/005,334.
Application 19/005,334 is a continuation of application No. 19/003,447, filed on Dec. 27, 2024.
Claims priority of provisional application 63/738,956, filed on Dec. 26, 2024.
Int. Cl. H04L 51/02 (2022.01); G10L 13/00 (2006.01); H04W 4/40 (2018.01)
CPC H04L 51/02 (2013.01) [G10L 13/00 (2013.01); H04W 4/40 (2018.02)] 27 Claims
OG exemplary drawing
 
1. A method implemented by one or more processors, the method comprising:
monitoring sensor data associated with an autonomous vehicle or an environment of the autonomous vehicle, the sensor data being generated by one or more sensors of the autonomous vehicle;
determining, based on an instance of the sensor data, whether to initiate a conversation with a remote communication participant that is located remotely from the autonomous vehicle;
determining a type of the instance of the sensor data, the type of the instance of the sensor data being one of multiple disparate types of instances of the sensor data;
identifying, based on the type of the instance of the sensor data, the remote communication participant that is located remotely from the autonomous vehicle; and
in response to determining to initiate the conversation with the remote communication participant that is located remotely from the autonomous vehicle and based on the instance of the sensor data:
initiating the conversation with the remote communication participant; and
conducting the conversation with the remote communication participant, wherein conducting the conversation with the remote communication participant comprises:
processing, using a generative model, generative model input to generate generative model output, the generative model input including at least an indication of the instance of the sensor data;
determining, based on the generative model output, content to be rendered as part of the conversation with the remote communication participant; and
causing the content to be rendered at a computing device associated with the remote communication participant.