US 12,347,038 B2
Crack detection, assessment and visualization using deep learning with 3D mesh model
Zheng Yi Wu, Watertown, CT (US); Rony Kalfarisi, North Sumatera (ID); and Ken Soh, Singapore (SG)
Assigned to Bentley Systems, Incorporated, Exton, PA (US)
Filed by Bentley Systems, Incorporated, Exton, PA (US)
Filed on Sep. 22, 2020, as Appl. No. 17/027,829.
Prior Publication US 2022/0092856 A1, Mar. 24, 2022
Int. Cl. G06T 19/00 (2011.01); G01C 11/02 (2006.01); G01C 11/04 (2006.01); G06F 18/214 (2023.01); G06V 10/22 (2022.01); G06V 10/44 (2022.01); G06V 20/20 (2022.01)
CPC G06T 19/006 (2013.01) [G01C 11/02 (2013.01); G01C 11/04 (2013.01); G06F 18/2148 (2023.01); G06V 10/22 (2022.01); G06V 10/44 (2022.01); G06V 20/20 (2022.01); G06T 2219/004 (2013.01)] 9 Claims
OG exemplary drawing
 
1. A method comprising:
acquiring a set of two-dimensional (2D) images of infrastructure;
identifying surface cracks in the 2D images and segmenting the identified surface cracks in the 2D images, by a trained Mask Region-Based Convolutional Neural Network (Mask-RCNN) executing on one or more electronic devices, to produce segmented 2D images having pixels that are divided into crack pixels that are part of identified surface cracks, and non-crack pixels that are not part of the identified surface cracks;
extracting, by a photogrammetry process executing on the one or more electronic devices, geometry information from the segmented 2D images produced by the Mask-RCNN and generating a three-dimensional (3D) mesh model of the infrastructure from the geometry information;
extracting, by the photogrammetry process, textures from the segmented 2D images produced by the Mask-RCNN and draping the textures on the 3D mesh model as a texture cover to form a textured 3D mesh model of the infrastructure;
indicating the identified surface cracks in the textured 3D mesh model by showing the crack pixels in the texture cover of the textured 3D mesh model with a contrasting color;
displaying, in a user interface on a display screen of the one or more electronic devices, the textured 3D mesh model to provide a visualization of identified surface cracks;
determining, by an analysis process executing on the one or more electronic devices, one or more quantitative measures of identified surface cracks in pixels based on the texture cover of the textured 3D mesh model;
determining a conversion factor between a number of pixels in the texture cover of the textured 3D mesh model and a distance in the infrastructure;
converting the one or more quantitative measures of identified surface cracks in pixels to one or more quantitative measures for the infrastructure by applying the conversion factor; and
displaying, in a user interface on a display screen, an indication of the one or more quantitative measures of identified surface cracks for the infrastructure.