These steps are described in more detail in the technical note
Surface-Based Matching. The generic parameters used to control these
steps are described in the respective sections below.
The further paragraphs describe the parameters and mention further points to
note.
The matching process and the parameters can be visualized and inspected
using the HDevelop procedure debug_find_surface_model.
points only. The normals are estimated based on the 3D neighborhood.
Note, this option is not recommended, since it generally leads to a
longer processing time and additionally the computed normals are usually
less accurate, leading to less accurate results.
It is important for an accurate PosePosePosePoseposepose that the normals of the scene
and the model point in the same direction (see
'scene_invert_normals'"scene_invert_normals""scene_invert_normals""scene_invert_normals""scene_invert_normals""scene_invert_normals").
If the model was trained for edge-supported surface-based matching and
the edge-supported matching has not been turned off via
'use_3d_edges'"use_3d_edges""use_3d_edges""use_3d_edges""use_3d_edges""use_3d_edges", only the
second combination is possible, i.e., the scene must contain a 2D mapping.
If the model was trained for edge-supported surface-based matching and
the scene contains a mapping, normals contained in the input point
cloud are not used (see 'scene_normal_computation'"scene_normal_computation""scene_normal_computation""scene_normal_computation""scene_normal_computation""scene_normal_computation" below).
Further, for models which were trained for edge-supported surface-based
matching it is necessary that the normal vectors point inwards.
Note that triangles or polygons in the passed scene are ignored.
Instead, only the vertices are used for matching.
It is thus in general not recommended to use this operator on meshed scenes,
such as CAD data.
Instead, such a scene must be sampled beforehand using
sample_object_model_3dsample_object_model_3dSampleObjectModel3dSampleObjectModel3dSampleObjectModel3dsample_object_model_3d to create points and normals
(e.g., using the method 'fast_compute_normals'"fast_compute_normals""fast_compute_normals""fast_compute_normals""fast_compute_normals""fast_compute_normals").
When using noisy point clouds, e.g., from time-of-flight cameras, the
generic parameter 'scene_normal_computation'"scene_normal_computation""scene_normal_computation""scene_normal_computation""scene_normal_computation""scene_normal_computation" could be set to
'mls'"mls""mls""mls""mls""mls" in order to obtain more robust results (see below).
ObjectModel3DObjectModel3DObjectModel3DObjectModel3DobjectModel3Dobject_model_3d is the handle of the 3D object model containing the
scene in which the matches are searched.
Note that in most cases, it is assumed the scene was observed from a camera
looking along the z-axis. This is important to align the scene normals if
they are re-computed (see 'scene_normal_computation'"scene_normal_computation""scene_normal_computation""scene_normal_computation""scene_normal_computation""scene_normal_computation" below).
In contrast, when the model was trained
for edge-supported surface-based matching and the scene contains a mapping,
normals are automatically aligned consistently.
The output parameter PosePosePosePoseposepose gives the 3D poses of the found object
instances. For every found instance of the surface model its pose is given
in the scene coordinate system, thus the pose is in the form
, where
scs denote the coordinate system of the scene (which often is
identical with the coordinate system of the sensor, the camera coordinate
system) and mcs the model coordinate system (which is a 3D world
coordinate system), see Transformations / Poses and
“Solution Guide III-C - 3D Vision”.
Thereby, the pose refers to the original coordinate system of the 3D object
model that was passed to create_surface_modelcreate_surface_modelCreateSurfaceModelCreateSurfaceModelCreateSurfaceModelcreate_surface_model.
The output parameter ScoreScoreScoreScorescorescore returns a score for each match.
Its value and interpretation differs for the cases distinguished below.
With pose refinement
For a matching with pose refinement, the score depends on whether
edge-support was activated:
Without edge-support, compute the surface fraction, i.e. the
approximate fraction of the object's surface that is visible in
the scene. This is done by counting the number of model points
that have a corresponding scene point and dividing this number
either by:
the total number of points on the model, if the
surface-based model is not prepared for view-based score
computation
or by:
the maximum number of potentially visible model points based
on the current viewpoint, if the surface-based model is prepared
for view-based score computation.
With edge-support, compute the geometric mean of the surface
fraction and the edge fraction. The surface fraction is affected
by whether the surface-based model is prepared for view-based score
computation or not, as explained above. The edge fraction is the number
of points from the sampled model edges that are aligned with
edges of the scene, divided by the maximum number of potentially
visible points of edges on the model.
Note that if the edges are extracted from multiple viewpoints, this
might lead to score greater than 1.
(if the scene was acquired from one single viewpoint)
(if the scene was merged from scenes that were acquired from N
different viewpoints)
Note that for the computation of the score after the sparse pose
refinement, the sampled scene points are used. For the
computation of the score after the dense pose refinement, all scene
points are used. Therefore, after the dense pose refinement, the
score values does not depend on the sampling distance of
the scene.
Without pose refinement
If only the first step, Approximate Matching, out of the three
steps described in The three steps of the matching takes place,
the possible score value and interpretation only differs whether there is
edge-support or not:
Without edge-support:
The score is the approximate number of points from the subsampled
scene that lie on the found object.
With edge-support:
The score is the approximate number of points from the subsampled
scene that lie on the found object multiplied with the number of points
from the sampled scene edges that are aligned with edges of the model.
For efficiency reasons, the maximum overlap can not be defined
in 3D. Instead, only the minimum distance between the centers
of the axis-aligned bounding boxes of two matches can be
specified with 'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel". The value is set
relative to the diameter of the object. Once an object with a
high ScoreScoreScoreScorescorescore is found, all other matches are suppressed
if the centers of their bounding boxes lie too close to the
center of the first object. If the resulting matches must
not overlap, the value for 'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel"
should be set to 1.0.
Note that only one of the parameters
'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel" and
'max_overlap_dist_abs'"max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs" should be set. If both are
set, only the value of the last modified parameter is used.
This parameter has the same effect as the parameter
'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel". Note that in contrast to
'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel", the value for
'max_overlap_dist_abs'"max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs" is set as an absolute value.
See 'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel" above, for a description of the
effect of this parameter.
Note that only one of the parameters
'max_overlap_dist_rel'"max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel""max_overlap_dist_rel" and
'max_overlap_dist_abs'"max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs""max_overlap_dist_abs" should be set. If both are
set, only the value of the last modified parameter is used.
This parameter controls the normal computation of the sampled
scene.
In the default mode 'fast'"fast""fast""fast""fast""fast", in most cases normals
from the 3D scene are used (if it already contains
normals) or computed based on
a small neighborhood of points (if not).
The computed normals are then oriented such that in case no original
normals exist.
This orientation of implies the assumption
that the scene was observed from a camera looking along the
z-axis.
In the default mode 'fast'"fast""fast""fast""fast""fast", in case the model was trained
for edge-supported surface-based matching and the scene contains a
mapping, input normals are not used and normals are always computed
from the mapping contained in the 3D scene. Further, the computed
normals are oriented inwards consistently with respect to the mapping.
Invert the orientation of the surface normals of the scene.
The orientation of surface normals of the scene have to match with the
orientation of the model.
If both the model and the scene are acquired with the same setup,
the normals will already point in the same direction.
If you experience the effect that the
model is found on the 'outside' of the scene surface, try to set this
parameter to 'true'"true""true""true""true""true".
Also, make sure that the normals in the scene all point either
outward or inward, i.e., are oriented consistently.
For edge-supported surface-based matching, the normal vectors have to
point inwards, but typically are automatically generated flipped inwards
consistently with respect to the mapping.
The orientation of the normals can be inspected using the procedure
debug_find_surface_model.
Possible values:'false'"false""false""false""false""false", 'true'"true""true""true""true""true"
Sets the threshold when extracting 3D edges for edge-supported
surface-based matching, i.e. if the surface model was created with
'train_3d_edges'"train_3d_edges""train_3d_edges""train_3d_edges""train_3d_edges""train_3d_edges" enabled.
The threshold is set relative to the diameter of the object.
Note that if edges were passed manually with the generic parameter
'3d_edges'"3d_edges""3d_edges""3d_edges""3d_edges""3d_edges", this parameter is ignored.
Otherwise, it behaves identically to the parameter
MinAmplitudeMinAmplitudeMinAmplitudeMinAmplitudeminAmplitudemin_amplitude of operator edges_object_model_3dedges_object_model_3dEdgesObjectModel3dEdgesObjectModel3dEdgesObjectModel3dedges_object_model_3d.
Similar to '3d_edge_min_amplitude_rel'"3d_edge_min_amplitude_rel""3d_edge_min_amplitude_rel""3d_edge_min_amplitude_rel""3d_edge_min_amplitude_rel""3d_edge_min_amplitude_rel", however, the value
is given as absolute distance and not relative to the object
diameter.
This parameter specifies the viewpoint from which the 3D data is
seen.
It is used for surface models
that are prepared for view-based score computation
(i.e. with 'train_view_based'"train_view_based""train_view_based""train_view_based""train_view_based""train_view_based" enabled) to get the maximum number
of potentially visible points of the model based on the current
viewpoint.
For this, GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value must contain a string consisting of
the three coordinates (x, y, and z) of the viewpoint, separated by
spaces. The viewpoint is defined in the same coordinate frame as
ObjectModel3DObjectModel3DObjectModel3DObjectModel3DobjectModel3Dobject_model_3d and should roughly correspond to the position the
scene was acquired from.
A visualization of the viewpoint can be created using the procedure
debug_find_surface_model in order to inspect its position.
Gaps in the 3D data are closed, as far as they do not exceed the
maximum gap size 'max_gap'"max_gap""max_gap""max_gap""max_gap""max_gap" [pixels] and the surface model
was created with 'train_3d_edges'"train_3d_edges""train_3d_edges""train_3d_edges""train_3d_edges""train_3d_edges" enabled.
Larger gaps will contain edges at their boundary, while gaps smaller
than this value will not.
This suppresses edges around smaller patches that were not
reconstructed by the sensor as well as edges at the more
distant part of a discontinuity.
For sensors with very large resolutions, the value should be
increased to avoid spurious edges.
Note that if edges were passed manually with the generic parameter
'3d_edges'"3d_edges""3d_edges""3d_edges""3d_edges""3d_edges", this parameter is ignored.
Otherwise, it behaves identically to the parameter
GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name of the operator edges_object_model_3dedges_object_model_3dEdgesObjectModel3dEdgesObjectModel3dEdgesObjectModel3dedges_object_model_3d
when 'max_gap'"max_gap""max_gap""max_gap""max_gap""max_gap" is set.
The influence of 'max_gap'"max_gap""max_gap""max_gap""max_gap""max_gap" can be inspected using the
procedure debug_find_surface_model.
Turns the edge-supported matching on or off.
This can be used to perform matching without 3D edges, even though
the model was created for edge-supported matching.
If the model was not created for edge-supported surface-based
matching, an error is returned.
List of values:'true'"true""true""true""true""true", 'false'"false""false""false""false""false"
In this second step, the approximate poses found in the previous step are
further refined. This increases the accuracy of the poses and the
significance of the score value.
Enables or disables the usage of scene normals for the pose
refinement. If this parameter is enabled, and if the scene contains
point normals, then those normals are used to increase the
accuracy of the pose refinement.
For this, the influence of scene points whose normal points in a
different direction than the model normal is decreased.
Note that the scene must contain point normals. Otherwise, this
parameter is ignored.
List of values:'true'"true""true""true""true""true", 'false'"false""false""false""false""false"
Turns the view-based score computation for surface-based matching on or
off. This can be used to perform matching without using the view-based
score, even though the model was prepared for view-based score
computation. The influence of 'use_view_based'"use_view_based""use_view_based""use_view_based""use_view_based""use_view_based" on the score is
explained in the documentation of ScoreScoreScoreScorescorescore above.
If the model was not prepared for view-based score computation,
an error is returned.
List of values:'true'"true""true""true""true""true", 'false'"false""false""false""false""false"
Default value:'false'"false""false""false""false""false", if 'train_view_based'"train_view_based""train_view_based""train_view_based""train_view_based""train_view_based"
was disabled when creating the model, otherwise 'true'"true""true""true""true""true".
3. Dense pose refinement
Accurately refines the poses found in the previous steps.
Number of iterations for the dense pose refinement. Increasing the
number of iteration leads to a more accurate pose at the expense of
runtime.
However, once convergence is reached, the accuracy can no longer be
increased, even if the number of steps is increased.
Note that this parameter is ignored if the dense pose refinement is
disabled.
Set the rate of scene points to be used for the dense pose
refinement.
For example, if this value is set to 5, every 5th point
from the scene is used for pose refinement. This parameter allows an
easy trade-off between speed and accuracy of the pose refinement:
Increasing the value leads to less points being used and in turn
to a faster but less accurate pose refinement. Decreasing the value
has the inverse effect.
Note that this parameter is ignored if the dense pose refinement is
disabled.
Set the distance threshold for dense pose refinement relative to the
diameter of the surface model.
Only scene points that are closer to the object than this distance
are used for the optimization. Scene points further
away are ignored.
Note that only one of the parameters
'pose_ref_dist_threshold_rel'"pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel"
and 'pose_ref_dist_threshold_abs'"pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Note that this parameter is ignored if the dense pose refinement is
disabled.
Set the distance threshold for dense pose refinement as an absolute
value.
See 'pose_ref_dist_threshold_rel'"pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel" for a detailed
description.
Note that only one of the parameters
'pose_ref_dist_threshold_rel'"pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel""pose_ref_dist_threshold_rel" and
'pose_ref_dist_threshold_abs'"pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs""pose_ref_dist_threshold_abs" should be set.
If both are set, only the value of the modified last parameter is used.
Set the distance threshold for scoring relative to the
diameter of the surface model. See the following
'pose_ref_scoring_dist_abs'"pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs" for a detailed description.
Note that only one of the parameters
'pose_ref_scoring_dist_rel'"pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel" and
'pose_ref_scoring_dist_abs'"pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Note that this parameter is ignored if the dense pose refinement is
disabled.
Set the distance threshold for scoring.
Only scene points that are closer to the object than this distance
are considered to be 'on the model' when computing the score after
the pose refinement. All other scene points are considered not to be
on the model.
The value should correspond to the amount of noise on the
coordinates of the scene points.
Note that this parameter is ignored if the dense pose refinement is
disabled.
Note that only one of the parameters
'pose_ref_scoring_dist_rel'"pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel""pose_ref_scoring_dist_rel" and
'pose_ref_scoring_dist_abs'"pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs""pose_ref_scoring_dist_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Enables or disables the usage of scene normals for the pose
refinement.
This parameter is explained in more details in the section
Sparse pose refinement above.
List of values:'true'"true""true""true""true""true", 'false'"false""false""false""false""false"
Set the distance threshold of edges for dense pose refinement relative
to the diameter of the surface model.
Only scene edges that are closer to the object edges than this distance
are used for the optimization. Scene edges further away are ignored.
Note that only one of the parameters
'pose_ref_dist_threshold_edges_rel'"pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel"
and 'pose_ref_dist_threshold_edges_abs'"pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Note that this parameter is ignored if the dense pose refinement is
disabled or if no edge-supported surface-based matching is used.
Set the distance threshold of edges for dense pose refinement as
an absolute value.
See 'pose_ref_dist_threshold_edges_rel'"pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel" for a detailed
description.
Note that only one of the parameters
'pose_ref_dist_threshold_edges_rel'"pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel""pose_ref_dist_threshold_edges_rel"
and 'pose_ref_dist_threshold_edges_abs'"pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs""pose_ref_dist_threshold_edges_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Note that this parameter is ignored if the dense pose refinement is
disabled or if no edge-supported surface-based matching is used.
Set the distance threshold of edges for scoring relative to the
diameter of the surface model. See the following
'pose_ref_scoring_dist_edges_abs'"pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs" for a detailed description.
Note that only one of the parameters
'pose_ref_scoring_dist_edges_rel'"pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel" and
'pose_ref_scoring_dist_edges_abs'"pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Note that this parameter is ignored if the dense pose refinement is
disabled or if no edge-supported surface-based matching is used.
Set the distance threshold of edges for scoring as an absolute value.
Only scene edges that are closer to the object edges than this distance
are considered to be 'on the model' when computing the score after
the pose refinement. All other scene edges are considered not to be
on the model.
The value should correspond to the expected inaccuracy of the extracted
scene edges and the inaccuracy of the refined pose.
Note that only one of the parameters
'pose_ref_scoring_dist_edges_rel'"pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel""pose_ref_scoring_dist_edges_rel" and
'pose_ref_scoring_dist_edges_abs'"pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs""pose_ref_scoring_dist_edges_abs" should be set.
If both are set, only the value of the last modified parameter is used.
Note that this parameter is ignored if the dense pose refinement is
disabled or if no edge-supported surface-based matching is used.
Turns the view-based score computation for surface-based matching on or
off. For further details, see the respective description in the section
about the sparse pose refinement above.
If the model was not prepared for view-based score computation,
an error is returned.
List of values:'true'"true""true""true""true""true", 'false'"false""false""false""false""false"
Default value:'false'"false""false""false""false""false", if 'train_view_based'"train_view_based""train_view_based""train_view_based""train_view_based""train_view_based"
was disabled when creating the model, otherwise 'true'"true""true""true""true""true".
Turns the optimization regarding self-similar, almost symmetric poses
on or off.
If the model was not created with activated parameter
'train_self_similar_poses'"train_self_similar_poses""train_self_similar_poses""train_self_similar_poses""train_self_similar_poses""train_self_similar_poses", an error is returned when setting
'use_self_similar_poses'"use_self_similar_poses""use_self_similar_poses""use_self_similar_poses""use_self_similar_poses""use_self_similar_poses" to 'true'"true""true""true""true""true".
List of values:'true'"true""true""true""true""true", 'false'"false""false""false""false""false"
Default value:'false'"false""false""false""false""false", if 'train_self_similar_poses'"train_self_similar_poses""train_self_similar_poses""train_self_similar_poses""train_self_similar_poses""train_self_similar_poses"
was disabled when creating the model, otherwise 'true'"true""true""true""true""true".
Execution Information
Multithreading type: reentrant (runs in parallel with non-exclusive operators).
Multithreading scope: global (may be called from any thread).
Automatically parallelized on internal data level.
This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.
This operator supports canceling timeouts and interrupts.
find_surface_modelfind_surface_modelFindSurfaceModelFindSurfaceModelFindSurfaceModelfind_surface_model returns 2 (
H_MSG_TRUE)
if all parameters are
correct. If necessary, an exception is raised.