CPC G16H 40/63 (2018.01) [G02B 27/017 (2013.01); G06F 3/167 (2013.01); G06N 3/04 (2013.01); G16H 30/40 (2018.01); G02B 2027/014 (2013.01)] | 19 Claims |
1. A non-transitory computer-readable medium storing instructions readable and executable by at least one electronic processor to perform an augmented reality (AR) content generation method, the method comprising:
acquiring, with a camera of an AR device, one or more images of a component of a medical imaging or medical therapy device;
processing the acquired one or more images with an image processor to generate one or more processed images;
receiving, from a microphone of the AR device, a triggering audio segment;
processing the received triggering audio segment with a speech-to-text process or a transcription process to generate a processed triggering audio segment;
generating one or more query vectors from both the one or more processed images and the processed triggering audio segment;
retrieving AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query vectors from a database; and
outputting the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker of the AR device.
|