US 11,986,963 B2
Augmented reality coordination of human-robot interaction
Daniel Szafir, Erie, CO (US); Michael E. Walker, Boulder, CO (US); and Hooman Hedayati, Rasht (IR)
Assigned to The Regents of the University of Colorado, Denver, CO (US)
Appl. No. 16/978,480
Filed by The Regents of the University of Colorado, Denver, CO (US)
PCT Filed Mar. 5, 2019, PCT No. PCT/US2019/020831
§ 371(c)(1), (2) Date Sep. 4, 2020,
PCT Pub. No. WO2019/173396, PCT Pub. Date Sep. 12, 2019.
Claims priority of provisional application 62/638,578, filed on Mar. 5, 2018.
Prior Publication US 2021/0094180 A1, Apr. 1, 2021
Int. Cl. B25J 13/06 (2006.01); B25J 9/16 (2006.01); G06F 3/14 (2006.01); G06T 11/00 (2006.01); G06V 20/10 (2022.01)
CPC B25J 9/1666 (2013.01) [B25J 13/06 (2013.01); G06F 3/14 (2013.01); G06T 11/00 (2013.01); G06V 20/10 (2022.01); G06T 2200/24 (2013.01)] 40 Claims
OG exemplary drawing
 
1. A method comprising:
receiving data collected by a robot collocated with a user wearing a head-mounted display,
wherein the data collected from the robot includes information about a local environment or objects within a past or current environment of the robot and one or more robot states;
localizing a coordinate system of a view of the head-mounted display relative to an orientation of the robot or an object within the past or current local environment of the robot within a line of sight of the user wearing the head-mounted display;
linking a user perspective of the environment to the data collected by the robot; and
updating, based on the data linked to the user perspective, an augmented reality or mixed reality view in the head-mounted display to integrate the user perspective with robot data.