US 11,861,898 B2
Self-expanding augmented reality-based service instructions library
Sven Peter Prevrhal, Hamburg (DE); and Jörg Sabczynski, Norderstedt (DE)
Assigned to KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
Appl. No. 16/757,921
Filed by KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
PCT Filed Oct. 18, 2018, PCT No. PCT/EP2018/078635
§ 371(c)(1), (2) Date Apr. 21, 2020,
PCT Pub. No. WO2019/081350, PCT Pub. Date May 2, 2019.
Claims priority of provisional application 62/575,553, filed on Oct. 23, 2017.
Prior Publication US 2020/0342228 A1, Oct. 29, 2020
Int. Cl. G06V 20/20 (2022.01); G06F 30/10 (2020.01); G06V 20/40 (2022.01); G06F 111/18 (2020.01)
CPC G06V 20/20 (2022.01) [G06F 30/10 (2020.01); G06V 20/40 (2022.01); G06F 2111/18 (2020.01)] 10 Claims
OG exemplary drawing
 
1. An augmented reality (AR) based servicing guidance device comprising:
a mobile device including a display, a processor, and a camera; and
a computer programmed to perform computer vision (CV) processing to generate AR content by operations including aligning at least a portion of the AR content with at least one location of interest (LOI) identified in a recorded video of the service call by a user input, wherein the CV processing comprises a Simultaneous Location and Mapping (SLAM) algorithm;
a server computer programmed to perform CV processing to align AR content with a live video feed captured by the camera of the mobile device, wherein the CV processing comprises a SLAM algorithm and wherein the aligned AR content includes a part number annotation, a computer aided design (CAD) drawing or CAD drawing portion, or a CAD animation aligned with a LOI identified in the live video feed by the CV processing; and
the processor of the mobile device is programmed to transmit the live video feed produced by the camera of the mobile device to the server computer and to receive the AR Content aligned with the live video feed from the sever computer and to display on the display of the mobile device the AR content aligned with the live video feed; and
the server computer is programmed to perform a servicing preview by performing the CV processing to align the AR content with preview video of how a step of the maintenance task is performed substituted for the live video feed produced by the camera of the mobile device and simultaneously displaying on the display of the mobile device both the AR content aligned with the preview video and the preview video displayed as an underlay of the AR content, the preview video displayed as an underlay of the AR content being a video to show how the maintenance steps are performed edited with the part number annotation, the CAD drawing or CAD drawing portion, or the CAD animation aligned with a LOI identified in the live video feed by the CV processing.