US 11,769,333 B2
Method and system for detecting block falling disaster in a tunnel based on visual and mechanical perception
Qi Jiang, Jinan (CN); and Qingyu Du, Jinan (CN)
Assigned to SHANDONG UNIVERSITY, Jinan (CN)
Filed by SHANDONG UNIVERSITY, Shandong (CN)
Filed on May 24, 2021, as Appl. No. 17/328,191.
Claims priority of application No. 202011130560.0 (CN), filed on Oct. 21, 2020.
Prior Publication US 2022/0121863 A1, Apr. 21, 2022
Int. Cl. G06V 20/58 (2022.01); G06T 7/62 (2017.01); G06T 7/73 (2017.01); G06T 7/246 (2017.01); G06V 20/64 (2022.01); G01M 5/00 (2006.01); H04N 7/18 (2006.01); G06F 18/214 (2023.01)
CPC G06V 20/58 (2022.01) [G01M 5/0025 (2013.01); G01M 5/0033 (2013.01); G01M 5/0075 (2013.01); G01M 5/0091 (2013.01); G06F 18/2148 (2023.01); G06T 7/251 (2017.01); G06T 7/62 (2017.01); G06T 7/75 (2017.01); G06V 20/653 (2022.01); H04N 7/188 (2013.01); G06T 2207/20084 (2013.01)] 16 Claims
OG exemplary drawing
 
5. A system for detecting a block falling disaster in a tunnel based on visual and mechanical perception, the system comprising a mobile platform and a control system, wherein:
a plurality of camera devices are arranged on the mobile platform for capturing tunnel inner wall images;
a positioning mechanism is arranged on the mobile platform for positioning the mobile platform;
a six-axis mechanical arm is arranged on the mobile platform, the six-axis mechanical arm is configured to move or stop at a specified position, and a six-dimensional force sensor is arranged at an end of the six-axis mechanical arm for acquiring force and torque; and
the control system is configured to receive the tunnel inner wall images and information of a displacement of the six-axis mechanical arm, the acquired force and torque about the end of the six-axis mechanical arm, and preliminarily identify a suspected block falling disaster using a trained first neural network model based on information of the images; further judging, using a second neural network model, the suspected block falling disaster based on the information of the displacement, force and torque, and finally determine the block falling disaster in the tunnel.