binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms (Operator)

Name

binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms — Compute the disparities of a rectified stereo image pair using multi-scanline optimization.

Signature

Description

binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms calculates the disparity between two rectified stereo images ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 using multi-scanline optimization. The resulting disparity image is returned in DisparityDisparityDisparityDisparitydisparitydisparity. In contrast to binocular_distance_msbinocular_distance_msBinocularDistanceMsBinocularDistanceMsBinocularDistanceMsbinocular_distance_ms, the results are not transformed into distance values.

For this task, the three operators binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity, binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMgbinocular_disparity_mg, and binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms can be used. binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity returns robust results in regions of sufficient texture but fails where is none. binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMgbinocular_disparity_mg interpolates low-texture regions but blurs discontinuities. binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms preserves discontinuities and interpolates partially.

binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms requires a reference image ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and a search image ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 which both must be rectified, i.e., corresponding pixels must have the same row coordinate. If this assumption is violated, the images can be rectified by using the operators calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerasCalibrateCamerascalibrate_cameras, gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map, and map_imagemap_imageMapImageMapImageMapImagemap_image.

ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 can have different widths, but must have the same height. Given a pixel in ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1, the homologous pixel in ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 is selected by searching along the corresponding row in ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 and matching both pixels based on a similarity measure. The disparity is the number of pixels by which each pixel in ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 needs to be moved to reach the homologous pixel in ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2. The search space is confined by the minimum and maximum disparity values MinDisparityMinDisparityMinDisparityMinDisparityminDisparitymin_disparity and MaxDisparityMaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity. If the minimum and maximum disparity values are set to an empty tuple, they are automatically set to the maximal possible range for the given images ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2.

Similar to binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity, a coarse-to-fine scheme may be used to improve the runtime. The coarse-to-fine scheme is used by setting GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name to 'method'"method""method""method""method""method" and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to 'fast'"fast""fast""fast""fast""fast". The default method is 'accurate'"accurate""accurate""accurate""accurate""accurate". The coarse-to-fine method requires significantly less memory and is significantly faster than the 'accurate'"accurate""accurate""accurate""accurate""accurate" method, especially for large images or a large range of MinDisparityMinDisparityMinDisparityMinDisparityminDisparitymin_disparity and MaxDisparityMaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity. The coarse-to-fine scheme has the further advantage that it automatically estimates the range of MinDisparityMinDisparityMinDisparityMinDisparityminDisparitymin_disparity and MaxDisparityMaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity while traversing through the pyramid. As a consequence, neither MinDisparityMinDisparityMinDisparityMinDisparityminDisparitymin_disparity nor MaxDisparityMaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity need to be set. However, the generated disparity images are less accurate for the 'fast'"fast""fast""fast""fast""fast" method than for the default 'accurate'"accurate""accurate""accurate""accurate""accurate" approach. Especially at sharp disparity jumps the 'fast'"fast""fast""fast""fast""fast" method preserves discontinuities less accurately. The runtime can be even further reduced, by setting the 'method'"method""method""method""method""method" to 'very_fast'"very_fast""very_fast""very_fast""very_fast""very_fast". However, this approach makes numerous assumptions that may lead to a smoothing of the disparities at discontinuities. Per default, the number of levels of the coarse-to-fine scheme is estimated automatically. It is possible to set the number of levels explicitly by setting GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name to 'num_levels'"num_levels""num_levels""num_levels""num_levels""num_levels" and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to the number of levels.

The similarity measure can be specified by setting GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name to 'similarity_measure'"similarity_measure""similarity_measure""similarity_measure""similarity_measure""similarity_measure" and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to the desired value. For both options 'census_dense'"census_dense""census_dense""census_dense""census_dense""census_dense" (default) and 'census_sparse'"census_sparse""census_sparse""census_sparse""census_sparse""census_sparse", the similarity measure is based on the Census transform. A Census transformed image contains for every pixel information about the intensity topology within a support window around it. 'census_dense'"census_dense""census_dense""census_dense""census_dense""census_dense" uses a dense 9 x 7 pixels window. 'census_sparse'"census_sparse""census_sparse""census_sparse""census_sparse""census_sparse" uses a sparse 15 x 15 pixels window where only a subset of the pixels is evaluated. While 'census_dense'"census_dense""census_dense""census_dense""census_dense""census_dense" is more suitable for fine structures, 'census_sparse'"census_sparse""census_sparse""census_sparse""census_sparse""census_sparse" is more robust on low texture areas.

To calculate the disparities from the similarity measure, the intermediate results are optimized by a multi-scanline method. The optimization increases the robustness in low-texture areas without blurring discontinuities in the disparity image. The optimization is controlled by the parameters SurfaceSmoothingSurfaceSmoothingSurfaceSmoothingSurfaceSmoothingsurfaceSmoothingsurface_smoothing and EdgeSmoothingEdgeSmoothingEdgeSmoothingEdgeSmoothingedgeSmoothingedge_smoothing. SurfaceSmoothingSurfaceSmoothingSurfaceSmoothingSurfaceSmoothingsurfaceSmoothingsurface_smoothing controls the smoothness within surfaces. High values suppress disparity differences of one pixel. EdgeSmoothingEdgeSmoothingEdgeSmoothingEdgeSmoothingedgeSmoothingedge_smoothing controls the occurrence and the shape of edges. Low values allow many edges, high values lead to fewer and rounder edges. For both parameters, reasonable values usually range between 0 and 100. If both parameters are set to zero, no optimization is performed.

The resulting disparity is returned in the single-channel image DisparityDisparityDisparityDisparitydisparitydisparity. A quality measure for each disparity value is returned in ScoreScoreScoreScorescorescore, containing the best (lowest) result of the optimized similarity measure of a reference pixel.

Furthermore, there are two optional post-processing steps. A consistency check is applied if GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name is set to 'consistency_check'"consistency_check""consistency_check""consistency_check""consistency_check""consistency_check" and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value is set to 'true'"true""true""true""true""true" (default). This increases the robustness of the returned matches since the result relies on a concurrent direct and reverse match. It is switched off by setting GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to 'false'"false""false""false""false""false".

Sub-pixel refinement of disparities is performed by setting GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name to 'sub_disparity'"sub_disparity""sub_disparity""sub_disparity""sub_disparity""sub_disparity" and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to 'true'"true""true""true""true""true" (default). It is switched off by setting GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to 'false'"false""false""false""false""false".

Finally, there are two optional parameters that control the coarse-to-fine scheme. If GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name is set to 'num_levels'"num_levels""num_levels""num_levels""num_levels""num_levels", the value of GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value determines the number of pyramids that are used. By setting GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to 'auto'"auto""auto""auto""auto""auto", the number of pyramids are automatically calculated. This is the default value of 'num_levels'"num_levels""num_levels""num_levels""num_levels""num_levels" and should be adequate for most applications. Furthermore, by setting GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name to 'disparity_offset'"disparity_offset""disparity_offset""disparity_offset""disparity_offset""disparity_offset", the quality of the coarse-to-fine approach at discontinuities can be adapted. The higher the value set in GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value, the more runtime is required. The default value is 3 and should be sufficient for most applications.

The runtime of the operator is approximately linear to the image width, the image height, and the disparity range. Consequently, the disparity range should be chosen as narrow as possible for large images. The runtime of the coarse-to-fine scheme (which is used by setting GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name to 'method'"method""method""method""method""method" and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value to 'fast'"fast""fast""fast""fast""fast" or 'very_fast'"very_fast""very_fast""very_fast""very_fast""very_fast") is approximately linear to the image width and the image height. For small images and small disparity ranges the runtime of the coarse-to-fine scheme may be larger than that of the 'accurate'"accurate""accurate""accurate""accurate""accurate" scheme.

Execution Information

• Supports OpenCL compute devices.
• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Automatically parallelized on tuple level.
• Automatically parallelized on internal data level.

Parameters

Example (HDevelop)

read_image (BaseballL, 'stereo/epipolar/baseball_l')
read_image (BaseballR, 'stereo/epipolar/baseball_r')
binocular_disparity_ms (BaseballL, BaseballR, Disparity, Score, \
                        -40, -10, 50, 50, [], [])

Result

If the parameter values are correct, binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms returns the value 2 (H_MSG_TRUE). If the input is empty (no input images are available) the behavior can be set via set_system('no_object_result',<Result>)set_system("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)set_system("no_object_result",<Result>). If necessary, an exception is raised.

Possible Predecessors

map_imagemap_imageMapImageMapImageMapImagemap_image

Possible Successors

thresholdthresholdThresholdThresholdThresholdthreshold, disparity_to_distancedisparity_to_distanceDisparityToDistanceDisparityToDistanceDisparityToDistancedisparity_to_distance, disparity_image_to_xyzdisparity_image_to_xyzDisparityImageToXyzDisparityImageToXyzDisparityImageToXyzdisparity_image_to_xyz

Alternatives

binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity, binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMgbinocular_disparity_mg, binocular_distancebinocular_distanceBinocularDistanceBinocularDistanceBinocularDistancebinocular_distance, binocular_distance_mgbinocular_distance_mgBinocularDistanceMgBinocularDistanceMgBinocularDistanceMgbinocular_distance_mg, binocular_distance_msbinocular_distance_msBinocularDistanceMsBinocularDistanceMsBinocularDistanceMsbinocular_distance_ms

See also

map_imagemap_imageMapImageMapImageMapImagemap_image, gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map, binocular_calibrationbinocular_calibrationBinocularCalibrationBinocularCalibrationBinocularCalibrationbinocular_calibration

Module

3D Metrology