find_component_modelT_find_component_modelFindComponentModelFindComponentModelfind_component_model (Operator)

Name

find_component_modelT_find_component_modelFindComponentModelFindComponentModelfind_component_model — Find the best matches of a component model in an image.

Signature

Description

The operator find_component_modelfind_component_modelFindComponentModelFindComponentModelFindComponentModelfind_component_model finds the best NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches instances of the component model ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id in the input image ImageImageImageImageimageimage.

The result of the search can be visualized using get_found_component_modelget_found_component_modelGetFoundComponentModelGetFoundComponentModelGetFoundComponentModelget_found_component_model. Also the operator can be used to extract the component matches of a certain component model instance.

Further information about input parameters

ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id:

Handle of the component model.

The model must have been created previously by calling create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model, or read in using read_component_modelread_component_modelReadComponentModelReadComponentModelReadComponentModelread_component_model.

RootComponentRootComponentRootComponentRootComponentrootComponentroot_component:

Index of the root component.

The components of the component model ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id are represented in in a tree structure. The component that stands at the root of this search tree is the root component.

The root component is searched within the full search space, i.e., at all allowed positions and in the allowed range of orientations (see below). In contrast, the remaining components are searched relative to the pose of their predecessor in the search tree within a restricted search space that is computed from the relations (recursive search).

To what extent a model component is suited to act as root component depends on several factors. In principle, a model component that can be found in the image with a high probability, should be chosen. Therefore, a component that is sometimes occluded to a high degree or that is missing in some cases is not well suited to act as root component. For the behavior in such cases, see the entry IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFoundif_root_not_found below.

A different possible criterion is the computation time that is associated with the root component during the search. A ranking of the model components that is based on the latter criterion is returned in RootRankingRootRankingRootRankingRootRankingrootRankingroot_ranking of the operator create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model, respectively. If the complete ranking is passed in RootComponentRootComponentRootComponentRootComponentrootComponentroot_component, the first value RootComponentRootComponentRootComponentRootComponentrootComponentroot_component[0] is automatically selected as the root component.

AngleStartRootAngleStartRootAngleStartRootAngleStartRootangleStartRootangle_start_root and AngleExtentRootAngleExtentRootAngleExtentRootAngleExtentRootangleExtentRootangle_extent_root:

Specify the allowed angle range (in [rad]) within which the root component is searched.

If necessary, the range of rotations is clipped to the range given when the component model was created with create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model, respectively. The angle range for each component can be queried with get_shape_model_paramsget_shape_model_paramsGetShapeModelParamsGetShapeModelParamsGetShapeModelParamsget_shape_model_params. The necessary handle of the corresponding shape model can be obtained using get_component_model_paramsget_component_model_paramsGetComponentModelParamsGetComponentModelParamsGetComponentModelParamsget_component_model_params.

MinScoreMinScoreMinScoreMinScoreminScoremin_score:

Determines what score a potential match of the component model must at least have to be regarded as an instance.

If the component model can be expected never to be occluded in the images, MinScoreMinScoreMinScoreMinScoreminScoremin_score may be set as high as 0.8 or even 0.9. The value of this parameter only slightly influences the computation time. An exception is the case of IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFoundif_root_not_found set to 'select_new_root'"select_new_root""select_new_root""select_new_root""select_new_root""select_new_root" (see below).

NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches:

Determines the maximum number of returned instances.

If fewer than NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches are found, only that number is returned, i.e., the parameter MinScoreMinScoreMinScoreMinScoreminScoremin_score takes precedence over NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches. In case more than NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches instances with a score greater than MinScoreMinScoreMinScoreMinScoreminScoremin_score are found in the image, only the best NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches instances are returned. However, if all model instances exceeding MinScoreMinScoreMinScoreMinScoreminScoremin_score in the image should be found, NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches must be set to 0.

MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlapmax_overlap:

Determines by what fraction two instances may at most overlap, whereby this fraction is a number between 0 and 1.

In some cases, found instances only differ in the pose of one or a few components. If two instances overlap each other by more than MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlapmax_overlap only the best instance is returned. This means, for MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlapmax_overlap = 0, the found instances may not overlap at all, while for MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlapmax_overlap = 1 no check for overlap is performed, and hence all instances are returned. The calculation of the overlap is based on the smallest enclosing rectangles of arbitrary orientation (see smallest_rectangle2smallest_rectangle2SmallestRectangle2SmallestRectangle2SmallestRectangle2smallest_rectangle2) of the found component instances.

IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFoundif_root_not_found:

specifies the behavior of the operator when dealing with a missing or strongly occluded root component.

Possible values:

• 'stop_search'"stop_search""stop_search""stop_search""stop_search""stop_search": It is assumed that the root component is always found in the image. Consequently, for instances for which the root component could not be found the search for the remaining components is not continued.

• 'select_new_root'"select_new_root""select_new_root""select_new_root""select_new_root""select_new_root": Different components are successively chosen as the root component and searched within the full search space. The order in which the selection of the root component is performed corresponds to the order passed in RootRankingRootRankingRootRankingRootRankingrootRankingroot_ranking. The poses of the found instances of all root components are then used to start the recursive search for the remaining components. Hence, it is possible to find instances even if the original root component is not found. However, the computation time of the search increases significantly in comparison to the search when choosing 'stop_search'"stop_search""stop_search""stop_search""stop_search""stop_search". This is especially the case for small values of MinScoreMinScoreMinScoreMinScoreminScoremin_score, as more root components must be searched. If during the search more root components are needed as given in RootComponentRootComponentRootComponentRootComponentrootComponentroot_component, the root components are completed by the automatically computed order (see create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model).

IfComponentNotFoundIfComponentNotFoundIfComponentNotFoundIfComponentNotFoundifComponentNotFoundif_component_not_found:

Specifies how components are searched when the predecessor component was not found (e.g., because she is missing or strongly occluded).

Possible values:

• 'prune_branch'"prune_branch""prune_branch""prune_branch""prune_branch""prune_branch": Such components are not searched at all and are also treated as 'not found'.

• 'search_from_upper'"search_from_upper""search_from_upper""search_from_upper""search_from_upper""search_from_upper": Such components are searched relative to the pose of the predecessor component of the predecessor component.

• 'search_from_best'"search_from_best""search_from_best""search_from_best""search_from_best""search_from_best": Such components are searched relative to the pose of the already found component from which the relative search can be performed with minimum computational effort.

PosePredictionPosePredictionPosePredictionPosePredictionposePredictionpose_prediction:

Determines whether the pose of not found components should be estimated.

Possible values:

• 'none'"none""none""none""none""none": Only the poses of the found components are returned.

• 'from_neighbors'"from_neighbors""from_neighbors""from_neighbors""from_neighbors""from_neighbors": The poses of not found components are estimated and returned with a score of ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp = 0.0. The pose estimation is based on the poses of the found neighboring components in the search tree.

• 'from_all'"from_all""from_all""from_all""from_all""from_all": The poses of not found components are estimated and returned with a score of ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp = 0.0. The pose estimation is based on the poses of all found components.

MinScoreCompMinScoreCompMinScoreCompMinScoreCompminScoreCompmin_score_comp:

Minimal necessary score of the components for the instances to be found.

This parameter has the same meaning as MinScoreMinScoreMinScoreMinScoreminScoremin_score in find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model.

Settable is either one element or the same number of elements as model components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id. In the first case the parameter is used for all components. In the second case, each parameter element refers to the corresponding component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id.

SubPixelCompSubPixelCompSubPixelCompSubPixelCompsubPixelCompsub_pixel_comp:

Determines whether the extraction shall be done subpixel precise and in the given case the maximal allowed object deformation in pixels.

This parameter has the same meaning as SubPixelSubPixelSubPixelSubPixelsubPixelsub_pixel in find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model. Therefore the maximal allowed object deformation has to be given as integer in the same string. Settable is either one element or the same number of elements as model components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id. In the first case the parameter is used for all components. In the second case, each parameter element refers to the corresponding component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id.

Example: ['least_squares', 'max_deformation 2']["least_squares", "max_deformation 2"]["least_squares", "max_deformation 2"]["least_squares", "max_deformation 2"]["least_squares", "max_deformation 2"]["least_squares", "max_deformation 2"].

NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsCompnum_levels_comp:

Determine the pyramid levels for the components used in the matching.

This parameter has the same meaning as NumLevelsNumLevelsNumLevelsNumLevelsnumLevelsnum_levels in find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model.

It determines the number of pyramid levels for the components to be used in the matching. Settable is either one element or the same number of elements as model components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id. In the first case the parameter is used for all components. In the second case, each parameter element refers to the corresponding component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id.

Optional, one can set value pairs for this parameter: In this case, the first value still determines the number of pyramid levels to be used. The second value specifies the lowest pyramid level, to which the found matches are tracked. In doing so, one can set either a single value pair or a value pair for each model component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id. If different value pairs should be used for different components, they must be specified in the same tuple. In case ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id contains exactly two components and in NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsCompnum_levels_comp two values are set, these values are interpreted as different number of pyramid levels to be used and not as a value pair.

Example: ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id contains two components, for which different pyramid levels shall be considered. For the first component 5 levels up to the level 2 shall be used. For the second component 4 levels up to the level 1 shall be used. In this case is NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsCompnum_levels_comp = [5,2,4,1].

GreedinessCompGreedinessCompGreedinessCompGreedinessCompgreedinessCompgreediness_comp:

“Greediness” of the search heuristic for the components: value from 0 to 1. Thereby 0 means: safe but slow, 1: fast but matches may be missed.

This parameter has the same meaning as GreedinessGreedinessGreedinessGreedinessgreedinessgreediness in find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model.

Settable is either one element or the same number of elements as model components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id. In the first case the parameter is used for all components. In the second case, each parameter element refers to the corresponding component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id.

Further information about output parameters

ModelStartModelStartModelStartModelStartmodelStartmodel_start and ModelEndModelEndModelEndModelEndmodelEndmodel_end:

Return the first and last index and therewith the index range of all component matches associated to the same instance of the component model.

The component matches corresponding to the first found instance of the component model are given by the interval of indices [ModelStartModelStartModelStartModelStartmodelStartmodel_start[0],ModelEndModelEndModelEndModelEndmodelEndmodel_end[0]]. Thereby the indices refer to the values of the parameters RowCompRowCompRowCompRowComprowComprow_comp, ColumnCompColumnCompColumnCompColumnCompcolumnCompcolumn_comp, AngleCompAngleCompAngleCompAngleCompangleCompangle_comp, ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp, and ModelCompModelCompModelCompModelCompmodelCompmodel_comp.

Example: The component model consists for three components. Two instances have been found on the image, where for one instance only two components (component 0 and component 2) could be found. Then the returned parameters could look like this:

RowCompRowCompRowCompRowComprowComprow_comp = [100,200,300,150,250]	ModelStartModelStartModelStartModelStartmodelStartmodel_start = [0,3]
ColumnCompColumnCompColumnCompColumnCompcolumnCompcolumn_comp = [200,210,220,400,425]	ModelEndModelEndModelEndModelEndmodelEndmodel_end = [2,4]
AngleCompAngleCompAngleCompAngleCompangleCompangle_comp = [0,0.1,-0.2,0.1,0.2]	ModelCompModelCompModelCompModelCompmodelCompmodel_comp = [0,1,2,0,2]
ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp = [1,1,1,1,1]	ScoreScoreScoreScorescorescore = [1,1]

From the right column it is visible, that in the left column:

• Values with index 0 to 2 correspond to the components 0 to 2 of instance 1.

• Values with index 3 to 4 correspond to the components 0 and 2 of instance 2.

ScoreScoreScoreScorescorescore:

Score of the found instances of the component model.

ScoreScoreScoreScorescorescore contains the weighted mean of the component scores, the values in ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp. The weighting is performed according to the number of model points within the respective component.

RowCompRowCompRowCompRowComprowComprow_comp, ColumnCompColumnCompColumnCompColumnCompcolumnCompcolumn_comp, and AngleCompAngleCompAngleCompAngleCompangleCompangle_comp:

The position (RowCompRowCompRowCompRowComprowComprow_comp, ColumnCompColumnCompColumnCompColumnCompcolumnCompcolumn_comp) and rotation (AngleCompAngleCompAngleCompAngleCompangleCompangle_comp) of the model components of all found component model instances.

The coordinates RowCompRowCompRowCompRowComprowComprow_comp and ColumnCompColumnCompColumnCompColumnCompcolumnCompcolumn_comp are the coordinates of the component origin (reference point) in the search image. The component origin depends on the model creation:

• with create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model by training: The component origin is the center of gravity of the respective returned contour region in ModelComponentsModelComponentsModelComponentsModelComponentsmodelComponentsmodel_components of the operator train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components.

• with create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model manually: The component origin is the center of gravity of the corresponding passed component region ComponentRegionsComponentRegionsComponentRegionsComponentRegionscomponentRegionscomponent_regions of the operator create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model.

Since the relations between the components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id refer to this reference point, the origin of the components must not be modified by using set_shape_model_originset_shape_model_originSetShapeModelOriginSetShapeModelOriginSetShapeModelOriginset_shape_model_origin.

ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp:

Score of each found component instance.

The score is a number between 0 and 1, and is an approximate measure of how much of the component is visible in the image. If, for example, half of the component is occluded, the score cannot exceed 0.5.

ModelCompModelCompModelCompModelCompmodelCompmodel_comp:

Index of the found component.

The tuple contains the indices of the respective model components (see create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model and train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components, respectively). By this the values in RowCompRowCompRowCompRowComprowComprow_comp, ColumnCompColumnCompColumnCompColumnCompcolumnCompcolumn_comp, AngleCompAngleCompAngleCompAngleCompangleCompangle_comp, and ScoreCompScoreCompScoreCompScoreCompscoreCompscore_comp can be associated to the different model components. See also the example given for ModelStartModelStartModelStartModelStartmodelStartmodel_start and ModelEndModelEndModelEndModelEndmodelEndmodel_end.

Information concerning the search

Internally, the shape-based matching is used for the component-based matching in order to search the individual components (see find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model).

The domain of the ImageImageImageImageimageimage determines the search space for the reference point, i.e., the allowed positions, of the root component.

Usually the component model is searched only within those points of the image domain in which the model fits completely into the image. This means that the components will not be found if they extend beyond the borders of the image, even if they would achieve a score greater than MinScoreCompMinScoreCompMinScoreCompMinScoreCompminScoreCompmin_score_comp (see above).

Note that, if for a certain pyramid level the component model touches the image border, it might not be found even if it lies completely within the original image. As a rule of thumb, the model might not be found if its distance to an image border falls below . This behavior can be changed with set_system('border_shape_models','true')set_system("border_shape_models","true")SetSystem("border_shape_models","true")SetSystem("border_shape_models","true")SetSystem("border_shape_models","true")set_system("border_shape_models","true"), which will cause components that extend beyond the image border to be found if they achieve a score greater than MinScoreCompMinScoreCompMinScoreCompMinScoreCompminScoreCompmin_score_comp. Here, points lying outside the image are regarded as being occluded, i.e., they lower the score. It should be noted that this mode increases the runtime of the search.

Note further, that in rare cases, which occur typically only for artificial images, the model might not be found also if for certain pyramid levels the model touches the border of the reduced domain. Then, it may help to enlarge the reduced domain by using, e.g., dilation_circledilation_circleDilationCircleDilationCircleDilationCircledilation_circle.

When tracking the matches through the image pyramid, on each level, some less promising matches are rejected based on NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches. Thus, it is possible that some matches are rejected that would have had a higher score on the lowest pyramid level. Due to this, for example, the found match for NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches set to 1 might be different from the match with the highest score returned when setting NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches to 0 or > 1.

Recommendations

If multiple objects with a similar score are expected, but only the one with the highest score should be returned, it might be preferable to raise NumMatchesNumMatchesNumMatchesNumMatchesnumMatchesnum_matches, and then select the match with the highest score.

To get a meaningful score value and to avoid erroneous matches, we recommend to always combine the allowance of a deformation with a subpixel extraction that applies a least-squares adjustment. If the subpixel extraction and/or the maximum object deformation is specified separately for each component, for each component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelIDcomponent_model_id exactly one value for the subpixel extraction must be passed in SubPixelCompSubPixelCompSubPixelCompSubPixelCompsubPixelCompsub_pixel_comp. After each value for the subpixel extraction optionally a second value can be passed, which describes the maximum object deformation of the corresponding mode. If for a certain component no value for the maximum object deformation is passed, the component is searched without taking deformations into account. Further details can be found in the documentation of find_shape_modelsfind_shape_modelsFindShapeModelsFindShapeModelsFindShapeModelsfind_shape_models.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Result

If the parameter values are correct, the operator find_component_modelfind_component_modelFindComponentModelFindComponentModelFindComponentModelfind_component_model returns the value 2 (H_MSG_TRUE). If the input is empty (no input image available) the behavior can be set via set_system('no_object_result',<Result>)set_system("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)set_system("no_object_result",<Result>). If necessary, an exception is raised.

Possible Predecessors

create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model, create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model, read_component_modelread_component_modelReadComponentModelReadComponentModelReadComponentModelread_component_model

Possible Successors

get_found_component_modelget_found_component_modelGetFoundComponentModelGetFoundComponentModelGetFoundComponentModelget_found_component_model

Alternatives

find_generic_shape_modelfind_generic_shape_modelFindGenericShapeModelFindGenericShapeModelFindGenericShapeModelfind_generic_shape_model

See also

find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model, find_shape_modelsfind_shape_modelsFindShapeModelsFindShapeModelsFindShapeModelsfind_shape_models, get_shape_model_paramsget_shape_model_paramsGetShapeModelParamsGetShapeModelParamsGetShapeModelParamsget_shape_model_params, get_component_model_paramsget_component_model_paramsGetComponentModelParamsGetComponentModelParamsGetComponentModelParamsget_component_model_params, train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components, set_shape_model_originset_shape_model_originSetShapeModelOriginSetShapeModelOriginSetShapeModelOriginset_shape_model_origin, smallest_rectangle2smallest_rectangle2SmallestRectangle2SmallestRectangle2SmallestRectangle2smallest_rectangle2

Module

Matching