Gantry detection problem can be simplified as line detection problem due to its special 2D features. Previous work on stereo vision-based gantry detection focus on the use of RGB image for gantry detection or line detection, and verify the “line” by its 3D depth information from stereo vision. However, this method does not fully take the advantage of stereo vision as it only uses depth information as a validation process. Some also directly apply 2D line detection techniques to normalised depth images. This will even add more noises to the detection as normalised depth maps are generally very blur.
Gantry detection flowchart
The algorithm proposed performs line detection in both RGB and depth images. Line detection in RGB image is based on Hough Line Transformation and line detection in depth image is based on morphological operation. Detection in depth image also takes the output from RGB line detection to generate an adaptive region of interest to increase the robustness of the algorithm.
In this project, I use RGB Detection as the abbreviation for line detection in RGB image and Depth Detection for line detection in depth image.
Most stereo vision system align the binocular camera horizontally, obtaining a stereo pair with horizontal disparity. Therefore, matching between corresponding pixels in two images is also performed in a horizontal way, such as Block Matching (BM) and Semi-global Block Matching (SGBM) algorithm. However, due to the matching ambiguities, the horizontal features are hard to reconstruct. If two images has similar pixel value in the same horizontal line, block matching based algorithms cannot well distinguish the corresponding pixels with the neighboring pixels. Hence decrease the matching accuracy. Generally gantries are placed horizontally and turns upright when it approaches. In order to detect the gantry in the horizontal position, the stereo vision camera in our system is mounted vertically onto the vehicle. The camera is allocated on the top-right corner to obtain a large point of view of the upright gantry on the right side when the vehicle approaches the gantry.
Stereo vision camera is mounted vertically on the right side of the vehicle
As discussed, the gantry detection algorithm comprises two parts:
- Hough Line Detection in RGB Image
- Morphological Operation in Depth Image
For Morphological Operation in Depth Image, the output image after horizontal line and vertical line extraction is shown as:
Morphological line extraction
Fusing the information, the final output is shown as:
From left to write is original image, RGB Detection and Depth Detection
Codes available at: https://github.com/Jin-Linhao/SMART_ZED
