US 12,235,899 B2
Sensor based semantic object generation
Tim Wantland, Bellevue, WA (US); Donald A. Barnett, Monroe, WA (US); and David Matthew Jones, London, NY (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Aug. 5, 2021, as Appl. No. 17/394,467.
Application 17/394,467 is a continuation of application No. 15/929,776, filed on May 21, 2020, granted, now 11,514,672.
Application 15/929,776 is a continuation of application No. 15/792,393, filed on Oct. 24, 2017, granted, now 10,685,233, issued on Jun. 16, 2020.
Prior Publication US 2021/0365684 A1, Nov. 25, 2021
Int. Cl. G06F 16/783 (2019.01); G06F 40/30 (2020.01); G06F 40/42 (2020.01); G06N 20/00 (2019.01); G06V 20/20 (2022.01); G06V 30/262 (2022.01); G06F 18/214 (2023.01)
CPC G06F 16/7837 (2019.01) [G06F 40/30 (2020.01); G06F 40/42 (2020.01); G06N 20/00 (2019.01); G06V 20/20 (2022.01); G06V 30/274 (2022.01); G06F 18/214 (2023.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for generating contextually-relevant actions, the method comprising:
providing, by a computing device, imagery or text for display within a first user interface, wherein the imagery is generated based on data captured by a camera integrated into the computing device;
recognizing, by the computing device, one or more semantic objects within the imagery or text displayed within the first user interface, wherein a respective semantic object in the one or more semantic objects has one or more attributes;
in response to recognition of the respective semantic objects object within the imagery or text displayed within the first user interface:
determining, by the computing device, an object type for the respective semantic object within the imagery display in the first user interface based on one or more attributes associated with the respective semantic object;
selecting, by the computing device, an interface element associated with the respective semantic object within the imagery display in the first user interface based on an object type;
providing, by the computing device, the interface element for display within a status bar, the status bar being visually separated from the first user interface;
receiving, by the computing device, a user interaction with the interface element within the status bar; and
in response to receipt of the user interaction with the interface element within the status bar: performing, by the computing device, an action related to the semantic object.