US 11,657,373 B2
System and method for identifying structural asset features and damage
Guanglei Xiong, Pleasanton, CA (US); Neeru Narang, San Jose, CA (US); Anwitha Paruchuri, San Jose, CA (US); Angela Yang Sanford, Oakland, CA (US); and Armando Ferreira Gomes, Austin, TX (US)
Assigned to Accenture Global Solutions Limited, Dublin (IE)
Filed by Accenture Global Solutions Limited, Dublin (IE)
Filed on Aug. 21, 2020, as Appl. No. 16/999,297.
Prior Publication US 2022/0058591 A1, Feb. 24, 2022
Int. Cl. G06Q 10/20 (2023.01); G06N 20/20 (2019.01); G06V 10/75 (2022.01); G06V 20/13 (2022.01); G06F 18/214 (2023.01); G06F 18/2113 (2023.01)
CPC G06Q 10/20 (2013.01) [G06F 18/214 (2023.01); G06F 18/2113 (2023.01); G06N 20/20 (2019.01); G06V 10/751 (2022.01); G06V 20/13 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method of detecting and identifying structural asset features and damage, comprising:
controlling an unmanned aerial vehicle (UAV) to fly to a first target site using one or more position sensors;
capturing, via the UAV, an image of a first target site;
receiving, from the UAV and at an artificial intelligence (AI) system, the image;
annotating, at the AI system, the image and generating an annotated image, the annotated image including at least one rotational bounding box surrounding a structural asset comprising a pole-mounted electrical utility device, wherein the rotational bounding box is oriented with at least two parallel lines running parallel to a first longitudinal axis of the structural asset, and the first longitudinal axis is diagonal relative to a second longitudinal axis of the annotated image;
receiving predefined metadata features of a first set of assets;
using a feature extraction model to:
extract identifying features from the annotated image defining color, edges, and texture, the feature extraction model comprising a first machine learning model trained using a first set of images to extract image-based features from the at least one rotational bounding box in the annotated image, and
extract pixel features related to individual pixels in the annotated image;
extract projection level features related to individual pixels in the annotated image defining scale, size, thickness, and orientation using a pre-built scale invariant feature transformation (SIFT) model; and
concatenating the identifying features, the pixel features, and the projection features extracted by the feature extraction model together with the predefined metadata features of the first set of assets through a fully connected layer and then processing the same through a Softmax layer to identify the structural asset and damage to the structural asset shown in the annotated image, and sending a repair crew to the first target site to repair the damaged structural asset.