US 11,727,683 B2
Methods and systems for image processing to present data in augmented reality
Matt D. McDole, St. Charles, MO (US); Sivaram Adhiappan, St. Charles, MO (US); Robert K. Gaddy, Wildwood, MO (US); Carlos G. Ruiz, St. Louis, MO (US); and Murat Sanli, Kirkwood, MO (US)
Assigned to Express Scripts Strategic Development, Inc., St. Louis, MO (US)
Filed by Express Scripts Strategic Development, Inc., St. Louis, MO (US)
Filed on Jun. 27, 2022, as Appl. No. 17/850,263.
Application 17/850,263 is a continuation of application No. 16/356,458, filed on Mar. 18, 2019, granted, now 11,373,400.
Prior Publication US 2022/0327823 A1, Oct. 13, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06V 20/20 (2022.01); G06T 19/00 (2011.01); G16H 80/00 (2018.01); G06K 7/14 (2006.01)
CPC G06V 20/20 (2022.01) [G06K 7/143 (2013.01); G06K 7/1443 (2013.01); G06T 19/006 (2013.01); G16H 80/00 (2018.01)] 19 Claims
OG exemplary drawing
 
1. A system comprising:
a camera;
a memory component;
a display communicatively coupled to the camera, the display configured to receive and present a live stream of images from within a field of view of the camera; and
a processor communicatively coupled to the camera, the memory component, and the display, the processor configured to:
(a) detect a glyph in the live stream of the images by the processor performing image analysis on the images;
(b) use the glyph to identify a current location of a container within the live stream of the images, wherein the glyph is contained on a sub region of the container;
(c) determine a first identifier associated with the container, by:
unwarping the live stream of the images;
in response to completing unwarping the live stream of the images, performing optical character recognition on at least one of the images from the camera; and
determining the first identifier based on the optical character recognition;
(d) determine whether a user has access to additional data about the container; and
(e) request the additional data about the container from a database; and
an augmented reality subsystem communicatively coupled to the camera, the memory component, the display, and the processor, the augmented reality subsystem configured to:
display the additional data about the container in an augmented reality foreground image, the additional data that is displayed including interactive buttons on the display, the interactive buttons configured to be activated to (i) transmit a request to refill a drug, (ii) call a doctor that prescribed the drug, and (iii) display drug interaction data about the drug, the augmented reality foreground image being displayed over the live stream of the images such that the additional data is displayed at a relative location adjacent to the current location and maintains the relative location as the current location changes in response to movement of the container or changes in the field of view of the camera.