US 12,316,959 B2
Environmental model maintenance using event-based vision sensors
Christian Brändli, Baden (CH); James G. Lewis, Riehen (CH); and Stefan Isler, Zürich (CH)
Assigned to Sony Advanced Visual Sensing AG, Schlieren (CH)
Appl. No. 17/421,167
Filed by Sony Advanced Visual Sensing AG, Schlieren (CH)
PCT Filed Jan. 25, 2020, PCT No. PCT/IB2020/050588
§ 371(c)(1), (2) Date Jul. 7, 2021,
PCT Pub. No. WO2020/152655, PCT Pub. Date Jul. 30, 2020.
Claims priority of provisional application 62/796,742, filed on Jan. 25, 2019.
Prior Publication US 2022/0060622 A1, Feb. 24, 2022
Int. Cl. H04N 23/65 (2023.01); G06F 1/3206 (2019.01); G06F 3/0346 (2013.01); G06F 11/30 (2006.01)
CPC H04N 23/65 (2023.01) [G06F 1/3206 (2013.01); G06F 3/0346 (2013.01); G06F 11/3058 (2013.01)] 22 Claims
OG exemplary drawing
 
1. A method for maintaining an environmental model of a computing device, comprising
monitoring an environment with one or more event-based vision sensors of the computing device in which the event-based vision sensors each comprises a two dimensional array of pixels in which each of the pixels of the event-based vision sensors includes a comparator that compares a difference between a photoreceptor signal and a past photoreceptor signal to an OFF-event threshold to determine OFF-events, which are discrete decreases in light intensity, and an ON-event threshold to determine ON-events, which are discrete increases in light intensity for the pixels;
acquiring motion information from an accelerometer and/or gyroscope of an inertial measurement unit;
based on the monitoring for the ON-events and OFF-events by the event-based vision sensor and the accelerometer and/or gyroscope, determining whether to adapt a control scheme of a depth sensor assembly and/or a camera of the computing device to acquire depth information and/or image information of the environment; and
updating the environmental model maintained by the computing device based on the depth information from the depth sensor assembly and/or image information from the camera.