US 12,392,903 B2
Computer-implemented method and system for generating synthetic sensor data, and training method
Daniel Hasenklever, Paderborn (DE)
Assigned to DSPACE GMBH, Paderborn (DE)
Appl. No. 17/436,082
Filed by DSPACE GMBH, Paderborn (DE)
PCT Filed Mar. 1, 2021, PCT No. PCT/EP2021/055048
§ 371(c)(1), (2) Date Jan. 12, 2022,
PCT Pub. No. WO2021/175783, PCT Pub. Date Sep. 10, 2021.
Claims priority of application No. 20160370 (EP), filed on Mar. 2, 2020.
Prior Publication US 2022/0326386 A1, Oct. 13, 2022
Int. Cl. G01S 17/931 (2020.01)
CPC G01S 17/931 (2020.01) 10 Claims
OG exemplary drawing
 
1. A method for generating synthetic sensor data corresponding to a LiDAR sensor of a vehicle, the synthetic sensor data including superimposed distance and intensity information, the method comprising:
providing a hierarchical variational autoencoder, wherein the hierarchical variational autoencoder has a first level of hierarchy and a second level of hierarchy, and wherein the hierarchical variational autoencoder has a third level of hierarchy or is configured to communicate with a third level of hierarchy of an external variational autoencoder;
receiving, by a variational autoencoder of the first level of hierarchy, a first data set of LiDAR sensor data including distance information, wherein the first data set comprises synthetically generated and/or captured real sensor data, the variational autoencoder of the first level of hierarchy assigning global features of the first data set to a first codebook vector;
receiving, by a variational autoencoder of the second level of hierarchy, the first data set, the variational autoencoder of the second level of hierarchy assigning local features of the first data set to a second codebook vector;
conditioning a first feature vector encoded by the variational autoencoder of the first level of hierarchy and a second feature vector encoded by the variational autoencoder of the second level of hierarchy with a second data set of LiDAR sensor data from the LiDAR sensor of the vehicle, the second data set including distance and intensity information;
combining the conditioned first feature vector and the conditioned second feature vector into a resulting third feature vector; and
decoding the resulting third feature vector to generate a third data set of synthetic LiDAR sensor data, the third data set including superimposed distance and intensity information;
wherein the first data set is encoded by a first encoder of the hierarchical variational autoencoder, wherein the encoding by the first encoder reduces an image resolution of the first data set;
wherein the first data set encoded by the first encoder is divided into the first level of hierarchy and the second level of hierarchy, wherein the first level of hierarchy of the first data set is further encoded by a second encoder of the hierarchical variational autoencoder, and wherein the encoding by the second encoder reduces the image resolution of the first data set;
wherein the first data set is encoded into the first feature vector by the second encoder. and wherein the first feature vector is assigned to the first codebook vector, which has the smallest distance to the first feature vector, by a first artificial convolutional neural network of the first level of hierarchy.