US 12,242,256 B2
Systems and methods for providing time-based user experiences for an industrial automation system
Thong T. Nguyen, New Berlin, WI (US); Paul D. Schmirler, Glendale, WI (US); Timothy T. Duffy, Franklin, WI (US); Kristopher J. Holley, Mequon, WI (US); and Susan J. Lovas, Sussex, WI (US)
Assigned to Rockwell Automation Technologies, Inc., Mayfield Heights, OH (US)
Filed by Rockwell Automation Technologies, Inc., Mayfield Heights, OH (US)
Filed on Sep. 22, 2021, as Appl. No. 17/482,165.
Prior Publication US 2023/0092405 A1, Mar. 23, 2023
Int. Cl. G05B 19/418 (2006.01); G06F 3/01 (2006.01); G06F 3/0482 (2013.01); G06F 16/23 (2019.01); G06T 11/00 (2006.01)
CPC G05B 19/41885 (2013.01) [G06F 3/011 (2013.01); G06F 3/0482 (2013.01); G06F 16/2379 (2019.01); G06T 11/00 (2013.01); G05B 2219/31229 (2013.01); G05B 2219/32014 (2013.01); G05B 2219/32337 (2013.01); G05B 2219/33034 (2013.01); G06T 2200/24 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A non-transitory computer-readable medium comprising instructions, wherein the instructions, when executed by processing circuitry, are configured to cause the processing circuitry to perform operations comprising:
receiving user input indicative of a selection of a user experience of a plurality of user experiences, wherein the plurality of user experiences comprises:
a first user experience associated with a first event that occurred in an industrial automation system during a first time period prior to receiving the user input; and
a second user experience associated with a second event expected to occur in the industrial automation system during a second time period after receiving the user input;
generating, based on the user input, a first output representative data or a second output representative data associated with the industrial automation system during the first time period or the second time period, respectively, wherein the first user experience is generated based on a first set of data acquired during the first time period by a plurality of sensors associated with the industrial automation system, wherein the first output representative data corresponds to a simulation of a user experience with respect to the industrial automation system during the first time period; and
wherein the second user experience is generated based on a predicted intent of a user determined by matching input sensory data and/or received context information with corresponding input sensory data and/or context information associated with the predicted intent, such that the second output representative data corresponds to an additional simulation of an expected user experience with respect to the industrial automation system during the second time period, and wherein the input sensory data is acquired by one or more sensors associated with the user, the industrial automation system, or both; and
instructing, based on the user input, a computing device to present the second output representative data in virtual reality, augmented reality, or both using image data and haptic feedback in response to receiving the user input indicative of the selection of the second user experience.