| CPC H04N 23/698 (2023.01) [G06T 7/50 (2017.01); G06T 7/70 (2017.01); G06T 17/20 (2013.01); G06V 10/16 (2022.01); G06T 2207/10024 (2013.01)] | 10 Claims |
|
1. A method of constructing a front panorama of shelving, the method comprising:
capturing, by at least one capturing device, at least two color image frames of shelving, each of the color image frames of the shelving displaying the shelving and shelving contents;
determining changes in a capturing position of the at least one capturing device from an initial capturing position of the at least one capturing device at a starting point of color image frames of the shelving;
determining a path of the at least one capturing device based on the change in the capturing position;
determining a depth of each captured color image frame of the shelving;
reconstructing a shelving 3D model as a 3D mesh based on the depth of each captured color image frame and the capturing position of the at least one capturing device for each color image frame of the shelving;
selecting from the 3D mesh only faces that are perpendicular to a horizontal plane and for which the following condition is satisfied:
n*g<ε,
y<ε,
where vector n(x,y,z) is the unit normal vector to a face, vector g(0,1,0) is the gravity vector, n*g is the scalar product of said vectors, ε is the tending to zero threshold of normal vector deviation;
approximating the path of the at least one capturing device to a first approximating straight line;
projecting the first approximating straight line onto the horizontal plane;
constructing, on the horizontal plane, perpendiculars to the projection of the first approximating straight line;
selecting vertices closest to the projection of the first approximating straight line, wherein the selected vertices fall on the perpendiculars when projected onto the horizontal plane;
selecting, from said selected vertices, a plurality of vertices, wherein projections of the plurality of vertices onto the horizontal plane are approximated by a second approximating straight line;
constructing, perpendicular to the horizontal plane, a projection plane passing through the second approximating straight line; and
stitching the at least two color image frames of the shelving by projective transformation of each color image frame of the shelving onto said projection plane to generate a shelving front panorama displaying the shelving and the shelving contents.
|