US 12,243,324 B2
Visual guidance system and method
Roya Khonsarian, Edmonton (CA); Amin Nazarzadeh Oghaz, Edmonton (CA); Bruce McGregor Alton, Edmonton (CA); and Siamak Akhlaghi Esfahany, Edmonton (CA)
Assigned to Correct-AI Inc., Edmonton (CA)
Filed by Correct-AI Inc., Edmonton (CA)
Filed on Aug. 26, 2021, as Appl. No. 17/446,099.
Claims priority of provisional application 63/071,665, filed on Aug. 28, 2020.
Prior Publication US 2022/0067403 A1, Mar. 3, 2022
Int. Cl. G06K 9/00 (2022.01); G05D 1/00 (2006.01); G06F 18/25 (2023.01); G06T 7/246 (2017.01); G06V 20/58 (2022.01)
CPC G06V 20/58 (2022.01) [G05D 1/0214 (2013.01); G06F 18/251 (2023.01); G06T 7/248 (2017.01); G05D 1/027 (2013.01); G05D 1/0278 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/30261 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A visual guidance system for a vehicle in an environment, the system comprising:
(a) an imaging system for producing a digital image of the environment;
(b) a three-dimensional (3D) scanning system for producing digital point cloud data of the environment;
(c) a system for monitoring the speed and direction of movement of the vehicle for generating data on the vehicle's speed and direction; and
(d) at least one processor coupled to the imaging system, 3D scanning system and the system for monitoring the vehicle's speed and direction, the at least one processor being configured for:
(i) processing the digital image to detect and classify target objects in the digital image, the target objects being one of a number of predefined object types requiring collision avoidance,
wherein objects that are not of the predefined object types are not detected and classified during processing of the digital image, and the digital image includes at least one object not of the predefined object types;
(ii) processing the digital point cloud data to group points into one or more point cloud groupings, each point cloud grouping representing a different object in the environment;
(iii) determining point cloud groupings associated with each of the target objects,
wherein the determination is based on identifying matching features between the digital point cloud values and the digital pixels for each of the target objects;
(iv) generating a fused data set frame by fusing only the point cloud grouping associated with the target objects with the corresponding digital image of the target objects, to produce a fused data set associated with each target object;
(v) determining each target object proximity from the vehicle based on the corresponding fused data set for that object in the fused data set frame;
(vi) determining each target object speed and direction by comparison of the fused data set frame including that fused data of that object, to a previously generated fused data set frame including fused data of the same object;
(vii) comparing each target objects' speed and direction to the vehicle's speed and direction to determine a likelihood of collision between the vehicle and each target object;
(viii) if a determination is made of collision, then determining that a given target object is an obstacle, and,
determining a threat level of the given target obstacle based on that object's speed and proximity from the vehicle;
determining if the threat level exceeds a predetermined threshold; and
generating an output if the threat level exceeds the predetermined threshold.