US 12,002,578 B2
Systems and methods for augmented reality-enhanced field services support
Rithesh Sreenivasan, Bangalore (IN); Oladimeji Feyisetan Farri, Yorktown Heights, NY (US); Sheikh Sadid Al Hasan, Cambridge, MA (US); Tilak Raj Arora, Gurgaon (IN); and Vivek Varma Datla, Ashland, MA (US)
Assigned to KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
Appl. No. 17/292,467
Filed by KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
PCT Filed Dec. 2, 2019, PCT No. PCT/EP2019/083207
§ 371(c)(1), (2) Date May 10, 2021,
PCT Pub. No. WO2020/120180, PCT Pub. Date Jun. 18, 2020.
Claims priority of provisional application 62/777,278, filed on Dec. 10, 2018.
Prior Publication US 2022/0020482 A1, Jan. 20, 2022
Int. Cl. G16H 40/63 (2018.01); G02B 27/01 (2006.01); G06F 3/16 (2006.01); G06N 3/04 (2023.01); G16H 30/40 (2018.01)
CPC G16H 40/63 (2018.01) [G02B 27/017 (2013.01); G06F 3/167 (2013.01); G06N 3/04 (2013.01); G16H 30/40 (2018.01); G02B 2027/014 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A non-transitory computer-readable medium storing instructions readable and executable by at least one electronic processor to perform an augmented reality (AR) content generation method, the method comprising:
acquiring, with a camera of an AR device, one or more images of a component of a medical imaging or medical therapy device;
processing the acquired one or more images with an image processor to generate one or more processed images;
receiving, from a microphone of the AR device, a triggering audio segment;
processing the received triggering audio segment with a speech-to-text process or a transcription process to generate a processed triggering audio segment;
generating one or more query vectors from both the one or more processed images and the processed triggering audio segment;
retrieving AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query vectors from a database; and
outputting the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker of the AR device.