US 12,354,197 B2
Interactive virtual graphics with physical objects
Jiahao Li, Los Angeles, CA (US); Li-Yi Wei, Redwood City, CA (US); Stephen DiVerdi, Berkeley, CA (US); and Kazi Rubaiat Habib, Seattle, WA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on May 10, 2022, as Appl. No. 17/740,653.
Prior Publication US 2023/0368452 A1, Nov. 16, 2023
Int. Cl. G06T 13/20 (2011.01); G06F 3/04815 (2022.01); G06F 3/04842 (2022.01); G06F 3/0486 (2013.01); G06T 7/70 (2017.01); G06T 19/00 (2011.01)
CPC G06T 13/20 (2013.01) [G06F 3/04815 (2013.01); G06F 3/04842 (2013.01); G06F 3/0486 (2013.01); G06T 7/70 (2017.01); G06T 19/006 (2013.01); G06T 2207/20044 (2013.01); G06T 2207/30204 (2013.01); G06T 2213/08 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
capturing a first image using a camera, the first image comprising an object in a first position, the object having a plurality of indicators indicating points of interest on the object;
displaying the first image;
receiving first user input, the first user input linking at least a subset of the indicators and establishing relationships between the points of interest on the object, wherein the first user input comprises an annotation associating a first indicator and a second indicator and establishing the relationships comprises, based on the first user input, establishing a portion of the object between the first indicator and the second indicator as a link to be tracked as a single rigid part;
receiving second user input, the second user input comprising a graphic element and a mapping between the graphic element and the object, including prompting and processing the second user input by:
identifying a point connecting the link and the graphic element;
establishing the point connecting the link and the graphic element as a connector;
displaying a set of user-selectable interface elements for configuring a connector type for the connector, each of the set of user-selectable interface elements corresponding to a respective selectable connector type;
receiving user input selecting a connector type; and
establishing a mechanical connection between the object and the graphic element based on the selected connector type;
capturing second images using the camera, the second images comprising the object in one or more modified positions;
tracking the one or more modified positions of the object across the second images using the indicators and the relationships between the points of interest on the object;
generating a virtual graphic based on the one or more modified positions of the object, the graphic element, and the mappings between the graphic element and the object; and
displaying a third image comprising the object with the virtual graphic.