US 12,265,702 B2
Systems and methods for efficient multimodal input collection with mobile devices
Donald A. Barnett, Monroe, WA (US); and Corbin Alexander Cunningham, Seattle, WA (US)
Assigned to Google LLC, Mountain View, CA (US)
Appl. No. 18/546,531
Filed by Google LLC, Mountain View, CA (US)
PCT Filed Apr. 28, 2021, PCT No. PCT/US2021/029599
§ 371(c)(1), (2) Date Aug. 15, 2023,
PCT Pub. No. WO2022/231581, PCT Pub. Date Nov. 3, 2022.
Prior Publication US 2024/0094895 A1, Mar. 21, 2024
Int. Cl. G06F 3/0488 (2022.01); G06F 21/32 (2013.01)
CPC G06F 3/0488 (2013.01) [G06F 21/32 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for contextualized input collection and intent determination for mobile devices, comprising:
providing, by a mobile computing system comprising one or more computing devices, a lock screen interface associated with the mobile computing system, wherein the lock screen interface comprises an input collection element configured to cause capture of sensor data while selected;
obtaining, by the mobile computing system from a user of the mobile computing system, an input signal that selects the input collection element;
in response to obtaining the input signal, capturing, by a plurality of sensors of the mobile computing system, the sensor data, wherein the plurality of sensors include an audio sensor and one or both of a front image sensor or a rear image sensor;
determining, by the mobile computing system based at least in part on content of the sensor data captured by the plurality of sensors including the audio sensor and the one or both of the front image sensor or the rear image sensor, one or more suggested action elements from a plurality of predefined action elements, wherein the one or more suggested action elements are respectively indicative of one or more device actions; and
providing for display, by the mobile computing system at a display device associated with the mobile computing system, the one or more suggested action elements within the lock screen interface.