US 12,246,736 B2
Image sensor for processing sensor data to reduce data traffic to host system
Poorna Kale, Folsom, CA (US); and Amit Gattani, Granite Bay, CA (US)
Assigned to Micron Technology, Inc., Boise, ID (US)
Filed by Micron Technology, Inc., Boise, ID (US)
Filed on Jul. 29, 2020, as Appl. No. 16/942,438.
Prior Publication US 2022/0032932 A1, Feb. 3, 2022
Int. Cl. B60W 50/04 (2006.01); G06F 13/42 (2006.01); G06F 18/214 (2023.01); G06F 18/24 (2023.01); G06N 3/08 (2023.01)
CPC B60W 50/045 (2013.01) [G06F 13/4282 (2013.01); G06F 18/2155 (2023.01); G06F 18/24 (2023.01); G06N 3/08 (2013.01); B60W 2420/408 (2024.01); B60W 2510/18 (2013.01); B60W 2510/20 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A system comprising:
a first image sensor configured to generate first image data;
at least one second sensor configured to generate sensor data in other than an image form, wherein the sensor data comprises sound data generated by at least one microphone;
at least one controller configured to convert the sound data to second image data using a Fourier transform, wherein the sound data is a time domain signal, and the sound data is converted to a two-dimensional image representation with each position of the two-dimensional image having a value corresponding to a frequency intensity;
a memory device configured to store the generated first image data, and a first portion of an artificial neural network (ANN); and
a host interface configured to communicate with a host system that stores a second portion of the ANN;
wherein the memory device is stacked with the first image sensor, and the memory device comprises an inference engine configured to generate inference results using the stored first image data as input to the first portion of the ANN, and further using the second image data as additional input to the first portion of the ANN;
wherein the host interface is further configured to send the inference results to the host system for processing by the host system using the second portion of the ANN;
wherein the host interface is further configured to receive a write command from the host system to store a third portion of the ANN in the memory device;
wherein the host system is further configured to send the third portion for storing in the memory device in response to evaluating prior data generated by the first image sensor using the second portion of the ANN, and further in response to determining that image frames are being dropped due to bandwidth limitations of communication between the host interface and the host system.