| CPC G06T 7/74 (2017.01) [G01C 21/005 (2013.01); G06T 1/20 (2013.01); G06T 7/13 (2017.01); G06T 7/73 (2017.01); G06T 2207/10032 (2013.01)] | 20 Claims |
|
1. A system implemented in a hardware circuit for descriptor-based feature matching during terrain relative navigation (TRN) comprising:
(a) a scale and orientation module comprising:
(i) a first memory read/write engine that feeds a first multi-stage computation pipeline wherein the first memory read/write engine:
(1) reads in image coordinates for a source image captured by a camera from a vehicle above a terrain of a scene;
(2) fetches an image pixel window from the source image around each of the image coordinates;
(3) fetches a slope pixel window from a slope image around each of the image coordinates, wherein the slope image is generated by a Harris corner detector, wherein the slope image is based on the source image;
(4) fetches a ring mask from a read only memory (ROM); and
(5) feeds the image pixel window, the slope pixel window, and the ring mask value to a first computation engine;
(ii) the first computation engine performing a plurality of primary stages in parallel in a pipeline fashion, wherein:
(1) a first primary stage of the plurality of primary stages comprises:
(A) combining corresponding pixels from the image pixel window and the slope pixel window with subwindow coordinates to generate intermediate values for each subwindow coordinate; and
(B) accumulating the intermediate values into ring accumulators based on the ring mask;
(2) a second primary stage of the plurality of primary stages comprises:
(A) reading the accumulated intermediate values from the ring accumulators; and
(B) summing the accumulated intermediate values for each ring accumulator to generate a final ring value for each ring accumulator;
(3) a third primary stage of the plurality of primary stages comprises multiple computations performed in parallel, and all of the computations are of a same latency, wherein:
(A) each computation is utilized to determine an orientation stability measure for each ring accumulator;
(B) each orientation stability measure is used to determine a final scale value and a final orientation value based on a threshold for the orientation stability measure;
(C) the final scale value is based on a first inner ring accumulator of the ring accumulators where the threshold was met; and
(D) the final orientation value is based on the accumulated intermediate values corresponding to the first inner ring accumulator where the threshold was met; and
(4) the final scale value and final orientation value are written in the first memory read/write engine;
(b) an extract descriptors (ED) module comprising:
(i) a second memory read/write engine that feeds a second multi-stage computation pipeline, wherein the second memory read/write engine:
(1) reads in the image coordinates for the source image captured by the camera from the vehicle above the terrain of the scene;
(2) fetches the image pixel window from the source image around each of the image coordinates;
(3) fetches the slope pixel window from the slope image around each of the image coordinates, wherein the slope image is generated by the Harris corner detector, wherein the slope image is based on the source image;
(4) reads the final scale value and the final orientation value for each of the image coordinates from external memory wherein the final scale value and the final orientation value were generated by the scale and orientation module;
(5) fetches sector values and a ring mask value from the ring mask from the read only memory (ROM);
(6) feeds the image pixel window, the slope pixel window, the sector values, the ring mask value, the final scale value, and the final orientation value to a second computation engine;
(7) identifies pixels of interest based on the final scale value and the ring mask value for a current image coordinate of the image coordinates; and
(8) reorients the sector values based on the final scale value and the final orientation value; and
(ii) the second computation engine performing two or more secondary stages in parallel in a pipeline fashion, wherein:
(1) a first secondary stage of the two or more secondary stages comprises:
(A) combining corresponding pixels from the image pixel window and the slope pixel window with subwindow coordinates to generate intermediate values for each subwindow coordinate; and
(B) accumulating the intermediate values into sector accumulators associated with the reoriented sector values; and
(2) additional secondary stages of the two or more secondary stages comprise:
(A) normalizing the intermediate values in the sector accumulators to generate an image feature descriptor per coordinate; and
(B) writing the image feature descriptors into external memory; and
wherein the image feature descriptors are used to perform the feature matching during the terrain relative navigation (TRN).
|