US 12,379,214 B2
Method of augmenting human perception of the surroundings
Remy Pottier, Grenoble (FR); Simon Roger Holland, Halam (GB); Eric Van Hensbergen, Austin, TX (US); and Christopher Daniel Emmons, Austin, TX (US)
Assigned to Arm Limited, Cambridge (GB)
Filed by Arm Limited, Cambridge (GB)
Filed on Mar. 28, 2022, as Appl. No. 17/705,796.
Prior Publication US 2023/0304800 A1, Sep. 28, 2023
Int. Cl. G01C 21/00 (2006.01); B60W 60/00 (2020.01); G01C 21/16 (2006.01); G06N 20/20 (2019.01); G06V 20/52 (2022.01)
CPC G01C 21/16 (2013.01) [B60W 60/0017 (2020.02); G01C 21/3848 (2020.08); G01C 21/3885 (2020.08); G06N 20/20 (2019.01); G06V 20/52 (2022.01)] 8 Claims
OG exemplary drawing
 
1. A computer-implemented method of augmenting human perception of the surroundings to identify a risk and/or an opportunity, comprising:
receiving a first set of data with respect to a user from a user device associated with the user, the first set of data comprising at least location information and a planned route, the user device comprising a display;
receiving a second set of data from a local distributed database associated with the location information, the second set of data comprising a position of one or more entities associated with the location information;
generating from the first set of data a respective motion vector for the user and generating from the second set of data a respective motion vector for the one or more entities, the motion vectors representing respective motion of the user and the one or more entities;
constructing a digital model representing the user and the one or more entities using the first and second set of data, wherein the digital model comprises digital twins of the user and the one or more entities and the respective motion vectors of the user and the one or more entities, the digital twins replicating the positions, motions and behavioural patterns of the user and the one or more entities;
projecting a respective course for the user and for the or each entity by inputting the digital model into a first machine learning algorithm (MLA) to infer one or more events involving the user in relation to the one or more entities;
identifying at least one relevant inferred event from the one or more inferred events by inputting the one or more inferred events to the first MLA to identify a risk or an opportunity relevant to the user;
communicating the at least one relevant inferred event to the user;
determining at least one appropriate response specific to the user based on the first set of data by inputting into the first MLA the at least one relevant inferred event; and
interacting with the user through the user device to assist the user in performing the at least one appropriate response, wherein interacting with the user comprises displaying the at least one relevant inferred event as a still image or a series of moving images on the user device and advising the user to take an alternative route.