proj_match_points_distortion_ransac_guidedT_proj_match_points_distortion_ransac_guidedProjMatchPointsDistortionRansacGuidedProjMatchPointsDistortionRansacGuided (Operator)

Name

proj_match_points_distortion_ransac_guidedT_proj_match_points_distortion_ransac_guidedProjMatchPointsDistortionRansacGuidedProjMatchPointsDistortionRansacGuided — Compute a projective transformation matrix and the radial distortion coefficient between two images by finding correspondences between points based on known approximations of the projective transformation matrix and the radial distortion coefficient.

Signature

Description

Given a set of coordinates of characteristic points (Rows1Rows1Rows1Rows1rows1,Cols1Cols1Cols1Cols1cols1) and (Rows2Rows2Rows2Rows2rows2Cols2Cols2Cols2Cols2cols2) in both input images Image1Image1Image1Image1image1 and Image2Image2Image2Image2image2, which must have identical size, and given known approximations HomMat2DGuideHomMat2DGuideHomMat2DGuideHomMat2DGuidehomMat2DGuide and KappaGuideKappaGuideKappaGuideKappaGuidekappaGuide for the transformation matrix and the radial distortion coefficient between Image1Image1Image1Image1image1 and Image2Image2Image2Image2image2, proj_match_points_distortion_ransac_guidedproj_match_points_distortion_ransac_guidedProjMatchPointsDistortionRansacGuidedProjMatchPointsDistortionRansacGuidedProjMatchPointsDistortionRansacGuided automatically determines corresponding points, the homogeneous projective transformation matrix HomMat2DHomMat2DHomMat2DHomMat2DhomMat2D, and the radial distortion coefficient KappaKappaKappaKappakappa that optimally fulfill the following equation:

The returned KappaKappaKappaKappakappa can be used to construct camera parameters that can be used to rectify images or points (see change_radial_distortion_cam_parchange_radial_distortion_cam_parChangeRadialDistortionCamParChangeRadialDistortionCamParChangeRadialDistortionCamPar, change_radial_distortion_imagechange_radial_distortion_imageChangeRadialDistortionImageChangeRadialDistortionImageChangeRadialDistortionImage, and change_radial_distortion_pointschange_radial_distortion_pointsChangeRadialDistortionPointsChangeRadialDistortionPointsChangeRadialDistortionPoints):

The approximations HomMat2DGuideHomMat2DGuideHomMat2DGuideHomMat2DGuidehomMat2DGuide and KappaGuideKappaGuideKappaGuideKappaGuidekappaGuide can, for example, be calculated with proj_match_points_distortion_ransacproj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansac on lower resolution versions of Image1Image1Image1Image1image1 and Image2Image2Image2Image2image2. See the example below.

The matching process is based on characteristic points, which can be extracted with point operators like points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerPointsFoerstner or points_harrispoints_harrisPointsHarrisPointsHarrisPointsHarris. The matching itself is carried out in two steps: first, gray value correlations of mask windows around the input points in the first and the second image are determined and an initial matching between them is generated using the similarity of the windows in both images. Then, the RANSAC algorithm is applied to find the projective transformation matrix and radial distortion coefficient that maximizes the number of correspondences under the above constraint.

The size of the mask windows used for the matching is MaskSizeMaskSizeMaskSizeMaskSizemaskSize x MaskSizeMaskSizeMaskSizeMaskSizemaskSize. Three metrics for the correlation can be selected. If GrayMatchMethodGrayMatchMethodGrayMatchMethodGrayMatchMethodgrayMatchMethod has the value 'ssd'"ssd""ssd""ssd""ssd", the sum of the squared gray value differences is used, 'sad'"sad""sad""sad""sad" means the sum of absolute differences, and 'ncc'"ncc""ncc""ncc""ncc" is the normalized cross correlation. For details please refer to binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparity. The metric is minimized ('ssd'"ssd""ssd""ssd""ssd", 'sad'"sad""sad""sad""sad") or maximized ('ncc'"ncc""ncc""ncc""ncc") over all possible point pairs. A thus found matching is only accepted if the value of the metric is below the value of MatchThresholdMatchThresholdMatchThresholdMatchThresholdmatchThreshold ('ssd'"ssd""ssd""ssd""ssd", 'sad'"sad""sad""sad""sad") or above that value ('ncc'"ncc""ncc""ncc""ncc").

To increase the algorithm's performance, the search area for the match candidates is limited based on the approximate transformation specified by HomMat2DGuideHomMat2DGuideHomMat2DGuideHomMat2DGuidehomMat2DGuide and KappaGuideKappaGuideKappaGuideKappaGuidekappaGuide. Only points within a distance of DistanceToleranceDistanceToleranceDistanceToleranceDistanceTolerancedistanceTolerance around the point in Image2Image2Image2Image2image2 that is obtained when transforming a point in Image1Image1Image1Image1image1 via HomMat2DGuideHomMat2DGuideHomMat2DGuideHomMat2DGuidehomMat2DGuide and KappaGuideKappaGuideKappaGuideKappaGuidekappaGuide are considered for the matching.

After the initial matching has been completed, a randomized search algorithm (RANSAC) is used to determine the projective transformation matrix HomMat2DHomMat2DHomMat2DHomMat2DhomMat2D and the radial distortion coefficient KappaKappaKappaKappakappa. It tries to find the parameters that are consistent with a maximum number of correspondences. For a point to be accepted, the distance to its corresponding transformed point must not exceed the threshold DistanceThresholdDistanceThresholdDistanceThresholdDistanceThresholddistanceThreshold. Consequently, DistanceThresholdDistanceThresholdDistanceThresholdDistanceThresholddistanceThreshold should be smaller than DistanceToleranceDistanceToleranceDistanceToleranceDistanceTolerancedistanceTolerance.

The parameter EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod determines which algorithm is used to compute the projective transformation matrix. A linear algorithm is used if EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod is set to 'linear'"linear""linear""linear""linear". This algorithm is very fast and returns accurate results for small to moderate noise of the point coordinates and for most distortions (except for small distortions). For EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard", a mathematically optimal but slower optimization is used, which minimizes the geometric reprojection error. In general, it is preferable to use EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard".

The value ErrorErrorErrorErrorerror indicates the overall quality of the estimation procedure and is the mean symmetric euclidean distance in pixels between the points and their corresponding transformed points.

Point pairs consistent with the above constraints are considered to be corresponding points. Points1Points1Points1Points1points1 contains the indices of the matched input points from the first image and Points2Points2Points2Points2points2 contains the indices of the corresponding points in the second image.

The parameter RandSeedRandSeedRandSeedRandSeedrandSeed can be used to control the randomized nature of the RANSAC algorithm, and hence to obtain reproducible results. If RandSeedRandSeedRandSeedRandSeedrandSeed is set to a positive number, the operator returns the same result on every call with the same parameters because the internally used random number generator is initialized with RandSeedRandSeedRandSeedRandSeedrandSeed. If RandSeedRandSeedRandSeedRandSeedrandSeed = 0, the random number generator is initialized with the current time. In this case the results may not be reproducible.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Example (HDevelop)

Factor := 0.5
zoom_image_factor (Image1, Image1Zoomed, Factor, Factor, 'constant')
zoom_image_factor (Image2, Image2Zoomed, Factor, Factor, 'constant')
points_foerstner (Image1Zoomed, 1, 2, 3, 200, 0.3, 'gauss', 'true', \
                  Rows1, Cols1, _, _, _, _, _, _, _, _)
points_foerstner (Image2Zoomed, 1, 2, 3, 200, 0.3, 'gauss', 'true', \
                  Rows2, Cols2, _, _, _, _, _, _, _, _)
get_image_size (Image1Zoomed, Width, Height)
proj_match_points_distortion_ransac (Image1Zoomed, Image2Zoomed, \
                                     Rows1, Cols1, Rows2, Cols2, \
                                     'ncc', 10, 0, 0, Height, Width, \
                                     0, 0.5, 'gold_standard', 2, 0, \
                                     HomMat2D, Kappa, Error, \
                                     Points1, Points2)
hom_mat2d_scale_local (HomMat2D, Factor, Factor, HomMat2DGuide)
hom_mat2d_scale (HomMat2DGuide, 1.0/Factor, 1.0/Factor, 0, 0, \
                 HomMat2DGuide)
KappaGuide := Kappa*Factor*Factor
points_foerstner (Image1, 1, 2, 3, 200, 0.3, 'gauss', 'true', \
                  Rows1, Cols1, _, _, _, _, _, _, _, _)
points_foerstner (Image2, 1, 2, 3, 200, 0.3, 'gauss', 'true', \
                  Rows2, Cols2, _, _, _, _, _, _, _, _)
proj_match_points_distortion_ransac_guided (Image1, Image2, \
                                            Rows1, Cols1, \
                                            Rows2, Cols2, \
                                            'ncc', 10, \
                                            HomMat2DGuide, \
                                            KappaGuide, 5, 0.5, \
                                            'gold_standard', 2, 0, \
                                            HomMat2D, Kappa, \
                                            Error, Points1, Points2)
get_image_size (Image1, Width, Height)
CamParDist := ['area_scan_division',0.0,Kappa,1.0,1.0, \
               0.5*(Width-1),0.5*(Height-1),Width,Height]
change_radial_distortion_cam_par ('fixed', CamParDist, 0, CamPar)
change_radial_distortion_image (Image1, Image1, Image1Rect, \
                                CamParDist, CamPar)
change_radial_distortion_image (Image2, Image2, Image2Rect, \
                                CamParDist, CamPar)
concat_obj (Image1Rect, Image2Rect, ImagesRect)
gen_projective_mosaic (ImagesRect, MosaicImage, 1, 1, 2, HomMat2D, \
                       'default', 'false', MosaicMatrices2D)

Possible Predecessors

points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerPointsFoerstner, points_harrispoints_harrisPointsHarrisPointsHarrisPointsHarris

Possible Successors

vector_to_proj_hom_mat2d_distortionvector_to_proj_hom_mat2d_distortionVectorToProjHomMat2dDistortionVectorToProjHomMat2dDistortionVectorToProjHomMat2dDistortion, change_radial_distortion_cam_parchange_radial_distortion_cam_parChangeRadialDistortionCamParChangeRadialDistortionCamParChangeRadialDistortionCamPar, change_radial_distortion_imagechange_radial_distortion_imageChangeRadialDistortionImageChangeRadialDistortionImageChangeRadialDistortionImage, change_radial_distortion_pointschange_radial_distortion_pointsChangeRadialDistortionPointsChangeRadialDistortionPointsChangeRadialDistortionPoints, gen_binocular_proj_rectificationgen_binocular_proj_rectificationGenBinocularProjRectificationGenBinocularProjRectificationGenBinocularProjRectification, projective_trans_imageprojective_trans_imageProjectiveTransImageProjectiveTransImageProjectiveTransImage, projective_trans_image_sizeprojective_trans_image_sizeProjectiveTransImageSizeProjectiveTransImageSizeProjectiveTransImageSize, projective_trans_regionprojective_trans_regionProjectiveTransRegionProjectiveTransRegionProjectiveTransRegion, projective_trans_contour_xldprojective_trans_contour_xldProjectiveTransContourXldProjectiveTransContourXldProjectiveTransContourXld, projective_trans_point_2dprojective_trans_point_2dProjectiveTransPoint2dProjectiveTransPoint2dProjectiveTransPoint2d, projective_trans_pixelprojective_trans_pixelProjectiveTransPixelProjectiveTransPixelProjectiveTransPixel

Alternatives

proj_match_points_distortion_ransacproj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansac

See also

proj_match_points_ransacproj_match_points_ransacProjMatchPointsRansacProjMatchPointsRansacProjMatchPointsRansac, proj_match_points_ransac_guidedproj_match_points_ransac_guidedProjMatchPointsRansacGuidedProjMatchPointsRansacGuidedProjMatchPointsRansacGuided, hom_vector_to_proj_hom_mat2dhom_vector_to_proj_hom_mat2dHomVectorToProjHomMat2dHomVectorToProjHomMat2dHomVectorToProjHomMat2d, vector_to_proj_hom_mat2dvector_to_proj_hom_mat2dVectorToProjHomMat2dVectorToProjHomMat2dVectorToProjHomMat2d

References

Richard Hartley, Andrew Zisserman: “Multiple View Geometry in Computer Vision”; Cambridge University Press, Cambridge; 2003.
Olivier Faugeras, Quang-Tuan Luong: “The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene and Some of Their Applications”; MIT Press, Cambridge, MA; 2001.

Module

Matching