US 12,333,592 B2
Method, computer program product, and system for displaying augmented reality elements for navigating to a location of an item within a warehouse
Spencer Schack, Seattle, WA (US); Andrew Peters, Tiburon, CA (US); Aditya Godbole, San Francisco, CA (US); Dominic Cocchiarella, Berkeley, CA (US); and Brandon Leonardo, San Carlos, CA (US)
Assigned to Maplebear Inc., San Francisco, CA (US)
Filed by Maplebear Inc., San Francisco, CA (US)
Filed on Aug. 21, 2023, as Appl. No. 18/236,370.
Application 18/236,370 is a continuation of application No. 17/877,718, filed on Jul. 29, 2022, granted, now 11,783,400.
Prior Publication US 2024/0037630 A1, Feb. 1, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06Q 30/0601 (2023.01); G06Q 10/087 (2023.01); G06T 19/00 (2011.01)
CPC G06Q 30/0631 (2013.01) [G06Q 10/087 (2013.01); G06Q 30/0633 (2013.01); G06Q 30/0643 (2013.01); G06T 19/003 (2013.01); G06T 19/006 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising:
capturing location data describing a first location of a client device within a warehouse, wherein the location data comprises measurements by a location sensor of the client device, and wherein the warehouse is associated with a plurality of items included among an inventory of the warehouse;
accessing a layout of the warehouse, wherein the layout of the warehouse describes a set of locations within the warehouse associated with each item of the plurality of items;
identifying a route for navigating from the first location of the client device to a second location within the warehouse associated with a target item of the plurality of items based at least in part on the first location of the client device and the layout of the warehouse;
generating a set of augmented reality elements based on the captured location data, wherein the set of augmented reality elements comprise a set of instructions for navigating the identified route within the warehouse, wherein the generated set of augmented reality elements comprise arrows to be overlaid on a floor demonstrating a path for a user to follow to navigate to the second location;
causing the set of augmented reality elements to be displayed by the client device by:
continually receiving image data captured by the client device;
continually augmenting the received image data to include augmented reality elements, wherein augmenting the received image data comprises overlaying the arrows of the set of augmented reality elements onto a portion of the received image data depicting the floor; and
continually causing the client device to display the augmented image data with the included augmented reality elements;
capturing additional location data from the location sensor of the client device;
determining that the client device is at the second location based on the additional location data;
determining that the target item is not at the second location; and
alerting a user of the client device to a replacement item for the target item by:
capturing, at the client device, one or more of an image and a video of a set of additional items, wherein each additional item of the set of additional items has at least a threshold measure of similarity to the target item;
communicating a set of information identifying each additional item of the set of additional items to an additional client device associated with another user associated with an online concierge system;
receiving information identifying the replacement item from the additional client device, wherein the replacement item is included among the set of additional items;
generating an augmented reality element that calls attention to the replacement item, and
sending the augmented reality element to a display area of the client device.