US 12,260,573 B2
Adversarial approach to usage of lidar supervision to image depth estimation
Nadav Shaag, Jerusalem (IL)
Assigned to Mobileye Vision Technologies Ltd., Jerusalem (IL)
Filed by Mobileye Vision Technologies Ltd., Jerusalem (IL)
Filed on Mar. 18, 2022, as Appl. No. 17/698,344.
Claims priority of provisional application 63/164,700, filed on Mar. 23, 2021.
Prior Publication US 2022/0309693 A1, Sep. 29, 2022
Int. Cl. G06T 7/521 (2017.01); G01B 11/22 (2006.01); G01S 17/89 (2020.01); G01S 17/931 (2020.01); G06N 3/08 (2023.01)
CPC G06T 7/521 (2017.01) [G01B 11/22 (2013.01); G01S 17/89 (2013.01); G01S 17/931 (2020.01); G06N 3/08 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20076 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30252 (2013.01)] 26 Claims
OG exemplary drawing
 
1. A method, comprising:
generating, via a generator network, a dense predicted Light Detection and Ranging (LIDAR) depth image based upon receiving, as training data, inputs that include (i) a camera image that was generated via one or more cameras, and (ii) a corresponding sparse LIDAR depth image that was generated via one or more LIDAR sensors;
generating, via an adversary network, a per-pixel label existence probability image based upon receiving the dense predicted LIDAR depth image as an input, the per-pixel label existence probability image representing, for each pixel in the dense predicted LIDAR depth image, a probability with respect to whether a predetermined label was identified from a supervision stage of the generator network;
iteratively training in parallel (i) a generator model via the generator network based upon the dense predicted LIDAR depth image, and (ii) an adversary model via the adversary network based upon the per-pixel label existence probability image; and
deploying the generator model to a vehicle to perform LIDAR depth prediction using data generated via one or more sensors implemented in the vehicle.
 
8. A computing device, comprising:
a memory configured to store computer-readable instructions; and
one or more processors configured to execute the computer readable instructions stored in the memory to:
generate, via a generator network, a dense predicted Light Detection and Ranging (LIDAR) depth image based upon receiving, as training data, inputs that include (i) a camera image that was generated via one or more cameras, and (ii) a corresponding sparse LIDAR depth image that was generated via one or more LIDAR sensors;
generate, via an adversary network, a per-pixel label existence probability image based upon receiving the dense predicted LIDAR depth image as an input, the per-pixel label existence probability image representing, for each pixel in the dense predicted LIDAR depth image, a probability with respect to whether a predetermined label was identified from a supervision stage of the generator network; and
iteratively train in parallel (i) a generator model via the generator network based upon the dense predicted LIDAR depth image, and (ii) an adversary model via the adversary network based upon the per-pixel label existence probability image,
wherein the generator model is deployed in a vehicle to perform LIDAR depth prediction using data generated via one or more sensors implemented in the vehicle.
 
15. A non-transitory computer-readable medium having instructions stored thereon that, when executed by processing circuitry of a computing device, cause the computing device to:
generate, via a generator network, a dense predicted Light Detection and Ranging (LIDAR) depth image based upon receiving, as training data, inputs that include (i) a camera image that was generated via one or more cameras, and (ii) a corresponding sparse LIDAR depth image that was generated via one or more LIDAR sensors;
generate, via an adversary network, a per-pixel label existence probability image based upon receiving the dense predicted LIDAR depth image as an input, the per-pixel label existence probability image representing, for each pixel in the dense predicted LIDAR depth image, a probability with respect to whether a predetermined label was identified from a supervision stage of the generator network; and
iteratively train in parallel (i) a generator model via the generator network based upon the dense predicted LIDAR depth image, and (ii) an adversary model via the adversary network based upon the per-pixel label existence probability image,
wherein the generator model is deployed in a vehicle to perform LIDAR depth prediction using data generated via one or more sensors implemented in the vehicle.