US 12,243,177 B1
Systems and methods for using artificial intelligence with assistive bots in extended reality environments
Ethan Fieldman, Gainesville, FL (US)
Assigned to VR-EDU, Inc., Gainesville, FL (US)
Filed by VR-EDU, Inc., Gainesville, FL (US)
Filed on Sep. 5, 2023, as Appl. No. 18/461,392.
Int. Cl. G06T 19/00 (2011.01); G06T 13/40 (2011.01); G10L 15/18 (2013.01)
CPC G06T 19/006 (2013.01) [G06T 13/40 (2013.01); G10L 15/1815 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for assisting a user experiencing an extended reality environment comprising:
displaying an assistive avatar in the extended reality environment to the user with a display of an extended reality hardware device being used by the user, wherein the assistive avatar is automatically controlled in real time by software communicating with the extended reality hardware device and the assistive avatar is not under real time control of a human;
receiving an input from the user in the extended reality environment that provides an inquiry to the assistive avatar for additional information about either a virtual object or a real-world object that the user is holding, touching or pointing to in the extended reality environment, wherein the input further includes the user indicating the virtual object or the real-world object that the user is holding, touching or pointing to, and the additional information is not already present at such time and location in the extended reality environment when the user provides the inquiry to the assistive avatar;
retrieving a response to the inquiry; and
providing the response to the inquiry via the assistive avatar as an answer to the inquiry, wherein said retrieving a response includes submitting a request for information based on the inquiry being provided as text data to an artificial intelligence process that applies a large language model to the text data and then transmits the response that includes the additional information about the virtual object or real-world object.