match_essential_matrix_ransacT_match_essential_matrix_ransacMatchEssentialMatrixRansacMatchEssentialMatrixRansacmatch_essential_matrix_ransac (Operator)

Name

match_essential_matrix_ransacT_match_essential_matrix_ransacMatchEssentialMatrixRansacMatchEssentialMatrixRansacmatch_essential_matrix_ransac — Compute the essential matrix for a pair of stereo images by automatically finding correspondences between image points.

Signature

Description

Given a set of coordinates of characteristic points (Rows1Rows1Rows1Rows1rows1rows_1,Cols1Cols1Cols1Cols1cols1cols_1) and (Rows2Rows2Rows2Rows2rows2rows_2,Cols2Cols2Cols2Cols2cols2cols_2) in the stereo images Image1Image1Image1Image1image1image_1 and Image2Image2Image2Image2image2image_2 along with known internal camera parameters, specified by the camera matrices CamMat1CamMat1CamMat1CamMat1camMat1cam_mat_1 and CamMat2CamMat2CamMat2CamMat2camMat2cam_mat_2, match_essential_matrix_ransacmatch_essential_matrix_ransacMatchEssentialMatrixRansacMatchEssentialMatrixRansacMatchEssentialMatrixRansacmatch_essential_matrix_ransac automatically determines the geometry of the stereo setup and finds the correspondences between the characteristic points. The geometry of the stereo setup is represented by the essential matrix EMatrixEMatrixEMatrixEMatrixEMatrixematrix and all corresponding points have to fulfill the epipolar constraint.

The operator match_essential_matrix_ransacmatch_essential_matrix_ransacMatchEssentialMatrixRansacMatchEssentialMatrixRansacMatchEssentialMatrixRansacmatch_essential_matrix_ransac is designed to deal with a linear camera model. The internal camera parameters are passed by the arguments CamMat1CamMat1CamMat1CamMat1camMat1cam_mat_1 and CamMat2CamMat2CamMat2CamMat2camMat2cam_mat_2, which are 3x3 upper triangular matrices describing an affine transformation. The relation between a vector (X,Y,1), representing the direction from the camera to the viewed 3D space point and its (projective) 2D image coordinates (col,row,1) is:

Note the column/row ordering in the point coordinates which has to be compliant with the x/y notation of the camera coordinate system. The focal length is denoted by f, are scaling factors, s describes a skew factor and indicates the principal point. Mainly, these are the elements known from the camera parameters as used for example in calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerasCalibrateCamerascalibrate_cameras. Alternatively, the elements of the camera matrix can be described in a different way, see e.g. stationary_camera_self_calibrationstationary_camera_self_calibrationStationaryCameraSelfCalibrationStationaryCameraSelfCalibrationStationaryCameraSelfCalibrationstationary_camera_self_calibration. Multiplied by the inverse of the camera matrices the direction vectors in 3D space are obtained from the (projective) image coordinates. For known camera matrices the epipolar constraint is given by:

The matching process is based on characteristic points, which can be extracted with point operators like points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerPointsFoerstnerpoints_foerstner or points_harrispoints_harrisPointsHarrisPointsHarrisPointsHarrispoints_harris. The matching itself is carried out in two steps: first, gray value correlations of mask windows around the input points in the first and the second image are determined and an initial matching between them is generated using the similarity of the windows in both images. Then, the RANSAC algorithm is applied to find the essential matrix that maximizes the number of correspondences under the epipolar constraint.

The size of the mask windows is MaskSizeMaskSizeMaskSizeMaskSizemaskSizemask_size x MaskSizeMaskSizeMaskSizeMaskSizemaskSizemask_size. Three metrics for the correlation can be selected. If GrayMatchMethodGrayMatchMethodGrayMatchMethodGrayMatchMethodgrayMatchMethodgray_match_method has the value 'ssd'"ssd""ssd""ssd""ssd""ssd", the sum of the squared gray value differences is used, 'sad'"sad""sad""sad""sad""sad" means the sum of absolute differences, and 'ncc'"ncc""ncc""ncc""ncc""ncc" is the normalized cross correlation. For details please refer to binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity. The metric is minimized ('ssd'"ssd""ssd""ssd""ssd""ssd", 'sad'"sad""sad""sad""sad""sad") or maximized ('ncc'"ncc""ncc""ncc""ncc""ncc") over all possible point pairs. A thus found matching is only accepted if the value of the metric is below the value of MatchThresholdMatchThresholdMatchThresholdMatchThresholdmatchThresholdmatch_threshold ('ssd'"ssd""ssd""ssd""ssd""ssd", 'sad'"sad""sad""sad""sad""sad") or above that value ('ncc'"ncc""ncc""ncc""ncc""ncc").

To increase the speed of the algorithm, the search area for the matching operations can be limited. Only points within a window of points are considered. The offset of the center of the search window in the second image with respect to the position of the current point in the first image is given by RowMoveRowMoveRowMoveRowMoverowMoverow_move and ColMoveColMoveColMoveColMovecolMovecol_move.

If the second camera is rotated around the optical axis with respect to the first camera the parameter RotationRotationRotationRotationrotationrotation may contain an estimate for the rotation angle or an angle interval in radians. A good guess will increase the quality of the gray value matching. If the actual rotation differs too much from the specified estimate the matching will typically fail. In this case, an angle interval should be specified, and RotationRotationRotationRotationrotationrotation is a tuple with two elements. The larger the given interval the slower the operator is since the RANSAC algorithm is run over all angle increments within the interval.

After the initial matching is completed a randomized search algorithm (RANSAC) is used to determine the essential matrix EMatrixEMatrixEMatrixEMatrixEMatrixematrix. It tries to find the essential matrix that is consistent with a maximum number of correspondences. For a point to be accepted, the distance to its corresponding epipolar line must not exceed the threshold DistanceThresholdDistanceThresholdDistanceThresholdDistanceThresholddistanceThresholddistance_threshold.

The parameter EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method decides whether the relative orientation between the cameras is of a special type and which algorithm is to be applied for its computation. If EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method is either 'normalized_dlt'"normalized_dlt""normalized_dlt""normalized_dlt""normalized_dlt""normalized_dlt" or 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard""gold_standard" the relative orientation is arbitrary. Choosing 'trans_normalized_dlt'"trans_normalized_dlt""trans_normalized_dlt""trans_normalized_dlt""trans_normalized_dlt""trans_normalized_dlt" or 'trans_gold_standard'"trans_gold_standard""trans_gold_standard""trans_gold_standard""trans_gold_standard""trans_gold_standard" means that the relative motion between the cameras is a pure translation. The typical application for this special motion case is the scenario of a single fixed camera looking onto a moving conveyor belt. In order to get a unique solution in the correspondence problem the minimum required number of corresponding points is six in the general case and three in the special, translational case.

The essential matrix is computed by a linear algorithm if 'normalized_dlt'"normalized_dlt""normalized_dlt""normalized_dlt""normalized_dlt""normalized_dlt" or 'trans_normalized_dlt'"trans_normalized_dlt""trans_normalized_dlt""trans_normalized_dlt""trans_normalized_dlt""trans_normalized_dlt" is chosen. With 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard""gold_standard" or 'trans_gold_standard'"trans_gold_standard""trans_gold_standard""trans_gold_standard""trans_gold_standard""trans_gold_standard" the algorithm gives a statistically optimal result, and returns the covariance of the essential matrix CovEMatCovEMatCovEMatCovEMatcovEMatcov_emat as well. Here, 'normalized_dlt' and 'gold_standard' stand for direct-linear-transformation and gold-standard-algorithm respectively. Note, that in general the found correspondences differ depending on the deployed estimation method.

The value ErrorErrorErrorErrorerrorerror indicates the overall quality of the estimation procedure and is the mean Euclidian distance in pixels between the points and their corresponding epipolar lines.

Point pairs consistent with the mentioned constraints are considered to be in correspondences. Points1Points1Points1Points1points1points_1 contains the indices of the matched input points from the first image and Points2Points2Points2Points2points2points_2 contains the indices of the corresponding points in the second image.

For the operator match_essential_matrix_ransacmatch_essential_matrix_ransacMatchEssentialMatrixRansacMatchEssentialMatrixRansacMatchEssentialMatrixRansacmatch_essential_matrix_ransac a special configuration of scene points and cameras exists: if all 3D points lie in a single plane and additionally are all closer to one of the two cameras then the solution in the essential matrix is not unique but twofold. As a consequence both solutions are computed and returned by the operator. This means that the output parameters EMatrixEMatrixEMatrixEMatrixEMatrixematrix, CovEMatCovEMatCovEMatCovEMatcovEMatcov_emat and ErrorErrorErrorErrorerrorerror are of double length and the values of the second solution are simply concatenated behind the values of the first one.

The parameter RandSeedRandSeedRandSeedRandSeedrandSeedrand_seed can be used to control the randomized nature of the RANSAC algorithm, and hence to obtain reproducible results. If RandSeedRandSeedRandSeedRandSeedrandSeedrand_seed is set to a positive number the operator yields the same result on every call with the same parameters because the internally used random number generator is initialized with the RandSeedRandSeedRandSeedRandSeedrandSeedrand_seed. If RandSeedRandSeedRandSeedRandSeedrandSeedrand_seed = 0 the random number generator is initialized with the current time. In this case the results may not be reproducible.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Possible Predecessors

points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerPointsFoerstnerpoints_foerstner, points_harrispoints_harrisPointsHarrisPointsHarrisPointsHarrispoints_harris

Possible Successors

vector_to_essential_matrixvector_to_essential_matrixVectorToEssentialMatrixVectorToEssentialMatrixVectorToEssentialMatrixvector_to_essential_matrix

See also

match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransac, match_rel_pose_ransacmatch_rel_pose_ransacMatchRelPoseRansacMatchRelPoseRansacMatchRelPoseRansacmatch_rel_pose_ransac, stationary_camera_self_calibrationstationary_camera_self_calibrationStationaryCameraSelfCalibrationStationaryCameraSelfCalibrationStationaryCameraSelfCalibrationstationary_camera_self_calibration

References

Richard Hartley, Andrew Zisserman: “Multiple View Geometry in Computer Vision”; Cambridge University Press, Cambridge; 2003.
Olivier Faugeras, Quang-Tuan Luong: “The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene and Some of Their Applications”; MIT Press, Cambridge, MA; 2001.

Module

3D Metrology