US 12,462,491 B2
Adaptive image processing for augmented reality device
Thomas Muttenthaler, Vienna (AT); and Kai Zhou, Wiener Neudorf (AT)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Mar. 8, 2023, as Appl. No. 18/118,906.
Prior Publication US 2024/0303934 A1, Sep. 12, 2024
Int. Cl. G06T 3/40 (2024.01); G02B 27/01 (2006.01); G06T 19/00 (2011.01); G06V 10/25 (2022.01)
CPC G06T 19/006 (2013.01) [G02B 27/0172 (2013.01); G06T 3/40 (2013.01); G06V 10/25 (2022.01); G02B 2027/0178 (2013.01); G06V 2201/07 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a first input image captured using at least one camera of an augmented reality (AR) device;
determining a region of interest of the first input image, the region of interest of the first input image including an object that is being tracked using an object tracking system;
determining, for the first input image and based on one or more object tracking parameters, a crop-and-scale order of an image processing operation directed at the region of interest of the first input image, the crop-and-scale order being dynamically adjustable between a first order and a second order, and the one or more object tracking parameters comprising at least one of object tracking status data, an object motion prediction, an object position relative to the region of interest, an AR device motion prediction, an AR device frame bending estimation, one or more camera-display transformation values, or a margin padding value;
generating, via performing of the image processing operation, a first output image from the first input image;
accessing a second input image captured using the at least one camera, the second input image depicting the object and captured subsequent to capturing of the first input image;
automatically adjusting, for the second input image and based on the one or more object tracking parameters, the crop-and-scale order of the image processing operation such that the crop-and-scale order for the second input image differs from the crop-and-scale order for the first input image;
generating, via performing of the image processing operation according to the crop-and-scale order for the second input image, a second output image from the second input image; and
accessing, by the object tracking system, the first output image and the second output image to track the object.