US 11,989,977 B2
System and method for authoring human-involved context-aware applications
Karthik Ramani, West Lafayette, IN (US); Tianyi Wang, West Lafayette, IN (US); and Xun Qian, West Lafayette, IN (US)
Assigned to Purdue Research Foundation, West Lafayette, IN (US)
Filed by Purdue Research Foundation, West Lafayette, IN (US)
Filed on Jun. 30, 2021, as Appl. No. 17/363,365.
Claims priority of provisional application 63/046,253, filed on Jun. 30, 2020.
Prior Publication US 2021/0406528 A1, Dec. 30, 2021
Int. Cl. G06T 7/215 (2017.01); G06F 3/01 (2006.01); G06T 13/40 (2011.01); G06T 19/00 (2011.01); G06V 20/20 (2022.01); G06V 40/10 (2022.01); G06V 40/20 (2022.01)
CPC G06V 40/28 (2022.01) [G06F 3/011 (2013.01); G06T 7/215 (2017.01); G06T 13/40 (2013.01); G06T 19/006 (2013.01); G06V 20/20 (2022.01); G06V 40/103 (2022.01); G06V 40/23 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A method for authoring an automation in which an operation is performed responsive to human actions in an environment, the method comprising:
synchronously recording, with at least one sensor, (i) human motions of a user in an environment, the recorded human motions including a plurality of recorded poses of the user in the environment over a period of time, and (ii) values for a context attribute over the period of time;
displaying, on a display, a graphical user interface including, superimposed on the environment, (i) a graphical representation of the recorded human motions including a graphical representation of the user superimposed on the environment, the graphical representation of the user representing a position and pose of the user at a particular time in the recorded human motions and (ii) a graphical representation of the recorded values for the context attribute;
moving, with a processor, based on user inputs received from the user, the graphical representation of the user to represent a position and pose of the user at a different particular time in the recorded human motions;
defining, with a processor, a first event by identifying, based on user inputs received from the user, (i) a first segment of the recorded human motions that corresponds to the first event and (ii) a first value for the context attribute that corresponds to the first event; and
defining, with the processor, based on user inputs received from the user, a first operation to be performed in response to the first event occurring.