US 11,756,297 B2
Augmented realty based assistance system and method thereof
Vivek Kumar Varma Nadimpalli, Hyderabad (IN); and Gopichand Agnihotram, Bangalore (IN)
Assigned to Wipro Limited, Bangalore (IN)
Filed by Wipro Limited, Bangalore (IN)
Filed on Apr. 27, 2021, as Appl. No. 17/241,124.
Claims priority of application No. 202141008993 (IN), filed on Mar. 3, 2021.
Prior Publication US 2022/0284217 A1, Sep. 8, 2022
Int. Cl. G06K 9/00 (2022.01); G06V 20/40 (2022.01); G06N 3/08 (2023.01); G06N 3/04 (2023.01); G06F 3/01 (2006.01)
CPC G06V 20/41 (2022.01) [G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06V 20/46 (2022.01); G06F 3/011 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of providing assistance to a user using augmented reality (AR), the method comprising:
acquiring, by an AR based assistance system, a video stream and a set of data associated with a task being performed by a user, in real-time, using at least one of a camera and a sensor device, wherein the video stream comprises a plurality of sequential frames;
determining, by the AR based assistance system, a present state associated with the task based on the plurality of sequential frames of the video stream using an Artificial Neural Network (ANN) based action prediction model, wherein determining the present state comprises extracting features from each of the plurality of sequential frames and generating feature vectors corresponding to the extracted features;
determining, by the AR based assistance system, a plurality of scenarios and a plurality of events corresponding to the plurality of scenarios based on the video stream and the set of data, using an ANN based augmented intelligence model; and
determining, by the AR based assistance system, a plurality of sequential instructions required for assisting the user to accomplish the task, dynamically, based on the present state and the plurality of events associated with the task, wherein at least one of a rule-based engine and an ANN based instruction prediction model is used to determine the plurality of sequential instructions.