US 12,469,228 B2
Mixed reality workflow generation
Edward Sean Lloyd Rintel, Cambridge (GB); Prashant Vaidyanathan, Purley (GB); Paul Kerr Grant, Cambridge (GB); Neil Dalchau, Cambridge (GB); Eyal Ofek, Redmond, WA (US); and Payod Panda, Cambridge (GB)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Dec. 2, 2022, as Appl. No. 18/061,271.
Prior Publication US 2024/0185534 A1, Jun. 6, 2024
Int. Cl. G06T 19/00 (2011.01); G06F 3/01 (2006.01); G06F 3/04842 (2022.01); G06Q 10/0631 (2023.01); G06T 13/40 (2011.01); G06T 13/80 (2011.01); G06V 20/20 (2022.01)
CPC G06T 19/006 (2013.01) [G06F 3/011 (2013.01); G06F 3/017 (2013.01); G06F 3/04842 (2013.01); G06Q 10/06316 (2013.01); G06T 13/40 (2013.01); G06T 13/80 (2013.01); G06V 20/20 (2022.01); G06T 2200/24 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method of generating a mixed reality workflow, the method comprising:
identifying a series of tasks and generating an input task-to-object-mapping by analyzing data that defines a process performed by a first user interacting with objects in a first location, wherein the input task-to-object-mapping comprises, for each task in the series of tasks, a mapping of the task to an object used in the task;
determining a task-specific non-spatial characteristic of each object in the input task-to-object-mapping;
identifying a plurality of candidate objects in a second location, location data defining a position of each candidate object within the second location and a non-spatial characteristic of each candidate object by analyzing data received from the second location;
generating an output task-to-object mapping by mapping each object in the input task-to-object-mapping to a candidate object using the task-specific non-spatial characteristic of each object in the input task-to-object-mapping and the non-spatial characteristic of each candidate object;
generating a mapped workflow comprising a process path using the series of tasks, the location data, and the output task-to-object-mapping; and
outputting the mapped workflow to a device in the second location, wherein
the task-specific non-spatial characteristic of each object comprises a capability of a respective object that is used to perform a respective task, and
the task-specific non-spatial characteristic of each object further comprises an environmental constraint associated with the respective object and the respective task.