US 12,272,177 B2
Method and apparatus for constructing three-dimensional data set of pedestrian re-identification based on neural radiation field
Hongsheng Wang, Hangzhou (CN); Guang Chen, Hangzhou (CN); and Hujun Bao, Hangzhou (CN)
Assigned to ZHEJIANG LAB, Hangzhou (CN)
Filed by ZHEJIANG LAB, Hangzhou (CN)
Filed on Sep. 21, 2022, as Appl. No. 17/950,033.
Application 17/950,033 is a continuation of application No. PCT/CN2022/116174, filed on Aug. 31, 2022.
Claims priority of application No. 202210670964.1 (CN), filed on Jun. 15, 2022.
Prior Publication US 2023/0410560 A1, Dec. 21, 2023
Int. Cl. G06V 40/20 (2022.01); G06V 10/56 (2022.01); G06V 10/82 (2022.01); G06V 20/64 (2022.01)
CPC G06V 40/25 (2022.01) [G06V 10/56 (2022.01); G06V 10/82 (2022.01); G06V 20/64 (2022.01)] 9 Claims
OG exemplary drawing
 
1. A method for constructing a three-dimensional data set of a pedestrian re-identification based on a neural radiation field, comprising the following steps:
S1: capturing images of pedestrians to be entered by a group of cameras at different viewing angles;
S2: generating a three-dimensional spatial position point set by sampling through camera rays in the scenario, and converting observation directions of the cameras corresponding to the three-dimensional spatial position point set into three-dimensional Cartesian unit vectors;
S3: inputting, into a multi-layer sensor, the three-dimensional spatial position point set and the observation directions converted into the three-dimensional Cartesian unit vectors, to output corresponding densities and colors;
S4: accumulating, by using a neural volume rendering method, ray colors passing through each pixel into the images captured in step S1, including the following sub-steps:
S4-1: defining cumulative transparency rates of the camera rays by continuous integration, and generating definitions of the ray colors accordingly;
S4-2: estimating the ray colors by using a quadrature method, dividing near boundaries to far boundaries of the rays into N uniformly spaced intervals, and selecting discrete points by using a stratified sampling method;
S5: introducing position encoding and multi-level sampling to improve the quality of the images generated by the accumulation of ray colors in step S4, specifically:
S5-1: introducing position encoding: encoding the spatial positions of the points, and converting the three-dimensional vectors input into a neural network into specified dimensions, to increase the accuracy of the generated images;
S5-2: introducing multi-level sampling: first collecting a group of points by stratified sampling, preliminarily evaluating the neural network, generating a probability density function based on the output of the preliminarily evaluated neural network, then collecting points along each ray based on the probability density function, and combining the points sampled twice to evaluate the neural network more accurately; and
S6: labeling the generated images and storing the same in a data set.