US 11,967,023 B2
Method and system for use in colourisation of a point cloud
Pavel Vechersky, Australian Capital Territory (AU); Mark Cox, Australian Capital Territory (AU); Paulo Borges, Australian Capital Territory (AU); Thomas Lowe, Australian Capital Territory (AU); and Michael Bosse, Australian Capital Territory (AU)
Assigned to Commonwealth Scientific and Industrial Research Organisation, Australian Capital Territory (AU)
Filed by Commonwealth Scientific and Industrial Research Organisation, Australian Capital Territory (AU)
Filed on Jan. 14, 2022, as Appl. No. 17/576,771.
Application 17/576,771 is a continuation of application No. 17/049,312, granted, now 11,270,501, previously published as PCT/AU2019/050393, filed on Apr. 30, 2019.
Claims priority of application No. 2018901452 (AU), filed on May 1, 2018.
Prior Publication US 2022/0139033 A1, May 5, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 17/05 (2011.01); G06T 7/20 (2017.01); G06T 7/80 (2017.01); G06T 19/20 (2011.01)
CPC G06T 17/05 (2013.01) [G06T 7/20 (2013.01); G06T 7/80 (2017.01); G06T 19/20 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/30241 (2013.01); G06T 2219/2012 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method for use in colourising a three-dimensional (3D) point cloud of an environment, the method including in one or more electronic processing devices:
a) using geometric data captured by a mapping device to:
i) determine a 3D point cloud of the environment; and,
ii) determine a mapping device trajectory;
b) obtaining video data indicative of a video recording of the environment captured by a camera moving along a camera trajectory, the video data including a number of frames;
c) determining camera extrinsic parameters indicative of a transformation which transforms 3D points in a mapping device coordinate system to a camera coordinate system;
d) for each of a plurality of frames, the method includes:
i) determining timing information indicative of a time of capture of the frame;
ii) determining a position of the camera along the camera trajectory using the mapping device trajectory, the timing information and the camera extrinsic parameters;
iii) identifying at least some 3D points of the point cloud in the frame based at least in part on the position of the camera and determining a local point cloud that captures a scene in the immediate vicinity of the camera, wherein a visibility check is performed on the local point cloud to determine points in the local point cloud that are visible from the perspective of the camera for a given frame; and
iv) determining a candidate colour for the at least some of the 3D points using a colour of a corresponding pixel in the frame extracted from the video data; and,
e) assigning a colour to one or more points of the 3D point cloud using candidate colours obtained from multiple frames, wherein assigning the colour includes determining an angle between the viewpoint to a surface where each 3D point is located on, and assigning a colour using a colour corresponding to the 3D point where the angle is closest to 90 degrees.