CPC H04N 13/383 (2018.05) [G02B 27/0172 (2013.01); G02B 2027/0138 (2013.01)] | 23 Claims |
1. A stereo vision procurement apparatus for obtaining stereo visual data, comprising: a stereo RGB camera;
a depth sensor; and
an RGB-D fusion module,
a processor; and
a plurality of tracking devices to track movement of a subject,
wherein:
the processor is configured to process data from the tracking devices to form a plurality of sub-features,
each of said stereo RGB camera and said depth sensor are configured to provide pixel data corresponding to a plurality of pixels,
said RGB-D fusion module is configured to combine RGB pixel data from said stereo RGB camera and depth information pixel data from said depth sensor to form stereo visual pixel data (SVPD), and
said RGB-D fusion module is implemented in an FPGA (field-programmable gate array) and wherein said sub-features are combined by said FPGA to form a feature to track movements of the subject;
further comprising: a memory; and
wherein said processor is configured to perform a defined set of operations in response to receiving a corresponding instruction selected from an instruction set of codes, and said instruction set of codes include:
a first set of codes for operating said RGB-D fusion module to synchronize RGB pixel data and depth pixel data, and for creating a disparity map; and
a second set of codes for creating a point cloud from said disparity map and said depth pixel data;
wherein said disparity map is generated by executing a plurality of instructions, wherein said instructions comprise performing a matching cost computation by measuring a similarity of pixels in left and right images by producing a cost; aggregating said costs to form a 3-D costs map; performing disparity selection to generate a 2-D disparity map; and refining said 2-D disparity map to generate said disparity map.
|