find_component_modelT_find_component_modelFindComponentModelFindComponentModel (Operator)

Name

find_component_modelT_find_component_modelFindComponentModelFindComponentModel — Find the best matches of a component model in an image.

Signature

Description

The operator find_component_modelfind_component_modelFindComponentModelFindComponentModelFindComponentModel finds the best NumMatchesNumMatchesNumMatchesNumMatchesnumMatches instances of the component model ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID in the input image ImageImageImageImageimage. The model must have been created previously by calling create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModel, create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel, or read_component_modelread_component_modelReadComponentModelReadComponentModelReadComponentModel.

The components of the component model ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID are represented in in a tree structure. The component that stands at the root of this search tree (root component) is searched within the full search space, i.e., at all allowed positions and in the allowed range of orientations (see below). In contrast, the remaining components are searched relative to the pose of their predecessor in the search tree within a restricted search space that is computed from the relations (recursive search). The index of the root component can be passed in RootComponentRootComponentRootComponentRootComponentrootComponent. To what extent a model component is suited to act as root component depends on several factors. In principle, a model component that can be found in the image with a high probability, should be chosen. Therefore, a component that is sometimes occluded to a high degree or that is missing in some cases is not well suited to act as root component. The behavior of the operator when dealing with a missing or strongly occluded root component can be set with IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFound (see below). Also, the computation time that is associated with the root component during the search can serve as a criterion. A ranking of the model components that is based on the latter criterion is returned in RootRankingRootRankingRootRankingRootRankingrootRanking of the operator create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModel or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel, respectively. If the complete ranking is passed in RootComponentRootComponentRootComponentRootComponentrootComponent, the first value RootComponentRootComponentRootComponentRootComponentrootComponent[0] is automatically selected as the root component. The domain of the image ImageImageImageImageimage determines the search space for the reference point, i.e., the allowed positions, of the root component. The parameters AngleStartRootAngleStartRootAngleStartRootAngleStartRootangleStartRoot and AngleExtentRootAngleExtentRootAngleExtentRootAngleExtentRootangleExtentRoot specify the allowed angle range within which the root component is searched. If necessary, the range of rotations is clipped to the range given when the component model was created with create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModel or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel, respectively. The angle range for each component can be queried with get_shape_model_paramsget_shape_model_paramsGetShapeModelParamsGetShapeModelParamsGetShapeModelParams after requesting the corresponding shape model handles with get_component_model_paramsget_component_model_paramsGetComponentModelParamsGetComponentModelParamsGetComponentModelParams.

The position and rotation of the model components of all found component model instances are returned in RowCompRowCompRowCompRowComprowComp, ColumnCompColumnCompColumnCompColumnCompcolumnComp, and AngleCompAngleCompAngleCompAngleCompangleComp. The coordinates RowCompRowCompRowCompRowComprowComp and ColumnCompColumnCompColumnCompColumnCompcolumnComp are the coordinates of the origin (reference point) of the component in the search image. If the component model was created with create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModel by training, the origin of the component is the center of gravity of the respective returned contour region in ModelComponentsModelComponentsModelComponentsModelComponentsmodelComponents of the operator train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponents. Otherwise, if the component model was created manually with create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel, the origin of the component is the center of gravity of the corresponding passed component region ComponentRegionComponentRegionComponentRegionComponentRegioncomponentRegion of the operator create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel. Since the relations between the components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID refer to this reference point, the origin of the components must not be modified by using set_shape_model_originset_shape_model_originSetShapeModelOriginSetShapeModelOriginSetShapeModelOrigin.

Additionally, the score of each found component instance is returned in ScoreCompScoreCompScoreCompScoreCompscoreComp. The score is a number between 0 and 1, and is an approximate measure of how much of the component is visible in the image. If, for example, half of the component is occluded, the score cannot exceed 0.5. While ScoreCompScoreCompScoreCompScoreCompscoreComp represents the score of the instances of the single components, ScoreScoreScoreScorescore contains the score of the instances of the entire component model. More precisely, ScoreScoreScoreScorescore contains the weighted mean of the associated values of ScoreCompScoreCompScoreCompScoreCompscoreComp. The weighting is performed according to the number of model points within the respective component.

In order to assign the values in RowCompRowCompRowCompRowComprowComp, ColumnCompColumnCompColumnCompColumnCompcolumnComp, AngleCompAngleCompAngleCompAngleCompangleComp, and ScoreCompScoreCompScoreCompScoreCompscoreComp to the associated model component, the index of the model component (see create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel and train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponents, respectively) is returned in ModelCompModelCompModelCompModelCompmodelComp. Furthermore, for each found instance of the component model its associated component matches are given in ModelStartModelStartModelStartModelStartmodelStart and ModelEndModelEndModelEndModelEndmodelEnd. Thus, the matches of the components that correspond to the first found instance of the component model are given by the interval of indices [ModelStartModelStartModelStartModelStartmodelStart[0],ModelEndModelEndModelEndModelEndmodelEnd[0]]. The indices refer to the parameters RowCompRowCompRowCompRowComprowComp, ColumnCompColumnCompColumnCompColumnCompcolumnComp, AngleCompAngleCompAngleCompAngleCompangleComp, ScoreCompScoreCompScoreCompScoreCompscoreComp, and ModelCompModelCompModelCompModelCompmodelComp. Assume, for example, that two instances of the component model, which consists of three components, are found in the image, where for one instance only two components (component 0 and component 2) could be found. Then the returned parameters could, for example, have the following elements: RowCompRowCompRowCompRowComprowComp = [100,200,300,150,250], ColumnCompColumnCompColumnCompColumnCompcolumnComp = [200,210,220,400,425], AngleCompAngleCompAngleCompAngleCompangleComp = [0,0.1,-0.2,0.1,0.2,0], ScoreCompScoreCompScoreCompScoreCompscoreComp = [1,1,1,1,1], ModelCompModelCompModelCompModelCompmodelComp = [0,1,2,0,2], ModelStartModelStartModelStartModelStartmodelStart = [0,3], ModelEndModelEndModelEndModelEndmodelEnd = [2,4], ScoreScoreScoreScorescore = [1,1]. The operator get_found_component_modelget_found_component_modelGetFoundComponentModelGetFoundComponentModelGetFoundComponentModel can be used to visualize the result of the search and to extract the component matches of a certain component model instance.

The component model is searched within those points of the domain of the image in which the model lies completely within the image. This means that the components will not be found if they extend beyond the borders of the image, even if they would achieve a score greater than MinScoreCompMinScoreCompMinScoreCompMinScoreCompminScoreComp (see below). Note that, if for a certain pyramid level the component model touches the image border, it might not be found even if it lies completely within the original image. As a rule of thumb, the model might not be found if its distance to an image border falls below . This behavior can be changed with set_system('border_shape_models','true')set_system("border_shape_models","true")SetSystem("border_shape_models","true")SetSystem("border_shape_models","true")SetSystem("border_shape_models","true"), which will cause components that extend beyond the image border to be found if they achieve a score greater than MinScoreCompMinScoreCompMinScoreCompMinScoreCompminScoreComp. Here, points lying outside the image are regarded as being occluded, i.e., they lower the score. It should be noted that the runtime of the search will increase in this mode. Note further, that in rare cases, which occur typically only for artificial images, the model might not be found also if for certain pyramid levels the model touches the border of the reduced domain. Then, it may help to enlarge the reduced domain by using, e.g., dilation_circledilation_circleDilationCircleDilationCircleDilationCircle.

The parameter MinScoreMinScoreMinScoreMinScoreminScore determines what score a potential match of the component model must at least have to be regarded as an instance of the component model in the image. If the component model can be expected never to be occluded in the images, MinScoreMinScoreMinScoreMinScoreminScore may be set as high as 0.8 or even 0.9. If a missing or strongly occluded root component must be assumed, and hence IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFound is set to 'select_new_root'"select_new_root""select_new_root""select_new_root""select_new_root" (see below), the search is faster the larger MinScoreMinScoreMinScoreMinScoreminScore is chosen. Otherwise, the value of this parameter only slightly influences the computation time.

The maximum number of model instances to be found can be determined with NumMatchesNumMatchesNumMatchesNumMatchesnumMatches. If more than NumMatchesNumMatchesNumMatchesNumMatchesnumMatches instances with a score greater than MinScoreMinScoreMinScoreMinScoreminScore are found in the image, only the best NumMatchesNumMatchesNumMatchesNumMatchesnumMatches instances are returned. If fewer than NumMatchesNumMatchesNumMatchesNumMatchesnumMatches are found, only that number is returned, i.e., the parameter MinScoreMinScoreMinScoreMinScoreminScore takes precedence over NumMatchesNumMatchesNumMatchesNumMatchesnumMatches. If all model instances exceeding MinScoreMinScoreMinScoreMinScoreminScore in the image should be found, NumMatchesNumMatchesNumMatchesNumMatchesnumMatches must be set to 0.

When tracking the matches through the image pyramid, on each level, some less promising matches are rejected based on NumMatchesNumMatchesNumMatchesNumMatchesnumMatches. Thus, it is possible that some matches are rejected that would have had a higher score on the lowest pyramid level. Due to this, for example, the found match for NumMatchesNumMatchesNumMatchesNumMatchesnumMatches set to 1 might be different from the match with the highest score returned when setting NumMatchesNumMatchesNumMatchesNumMatchesnumMatches to 0 or > 1.

If multiple objects with a similar score are expected, but only the one with the highest score should be returned, it might be preferable to raise NumMatchesNumMatchesNumMatchesNumMatchesnumMatches, and then select the match with the highest score.

In some cases, found instances only differ in the pose of one or a few components. The parameter MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlap determines by what fraction (i.e., a number between 0 and 1) two instances may at most overlap in order to consider them as different instances, and hence to return them separately. If two instances overlap each other by more than MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlap only the best instance is returned. The calculation of the overlap is based on the smallest enclosing rectangles of arbitrary orientation (see smallest_rectangle2smallest_rectangle2SmallestRectangle2SmallestRectangle2SmallestRectangle2) of the found component instances. If MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlap = 0, the found instances may not overlap at all, while for MaxOverlapMaxOverlapMaxOverlapMaxOverlapmaxOverlap = 1 no check for overlap is performed, and hence all instances are returned.

The parameter IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFound specifies the behavior of the operator when dealing with a missing or strongly occluded root component. This parameter strongly influences the computation time of the operator. If IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFound is set to 'stop_search'"stop_search""stop_search""stop_search""stop_search", it is assumed that the root component is always found in the image. Consequently, for instances for which the root component could not be found the search for the remaining components is not continued. If IfRootNotFoundIfRootNotFoundIfRootNotFoundIfRootNotFoundifRootNotFound is set to 'select_new_root'"select_new_root""select_new_root""select_new_root""select_new_root", different components are successively chosen as the root component and searched within the full search space. The order in which the selection of the root component is performed corresponds to the order passed in RootRankingRootRankingRootRankingRootRankingrootRanking. The poses of the found instances of all root components are then used to start the recursive search for the remaining components. Hence, it is possible to find instances even if the original root component is not found. However, the computation time of the search increases significantly in comparison to the search when choosing 'stop_search'"stop_search""stop_search""stop_search""stop_search". The number of root components to search depends on the value specified for MinScoreMinScoreMinScoreMinScoreminScore. The higher the value for MinScoreMinScoreMinScoreMinScoreminScore is chosen the fewer root components must be searched, and thus the faster the search is performed. If the number of elements in RootComponentRootComponentRootComponentRootComponentrootComponent is less than the number of required root components during the search, the root components are completed by the automatically computed order (see create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModel or create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel).

The parameter IfComponentNotFoundIfComponentNotFoundIfComponentNotFoundIfComponentNotFoundifComponentNotFound specifies the behavior of the operator when dealing with missing or strongly occluded components other than the root component. Here, it can be stated in which way components that must be searched relative to the pose of another (predecessor) component should be treated if the predecessor component was not found. If IfComponentNotFoundIfComponentNotFoundIfComponentNotFoundIfComponentNotFoundifComponentNotFound is set to 'prune_branch'"prune_branch""prune_branch""prune_branch""prune_branch", such components are not searched at all and are also treated as 'not found'. If IfComponentNotFoundIfComponentNotFoundIfComponentNotFoundIfComponentNotFoundifComponentNotFound is set to 'search_from_upper'"search_from_upper""search_from_upper""search_from_upper""search_from_upper", such components are searched relative to the pose of the predecessor component of the predecessor component. If IfComponentNotFoundIfComponentNotFoundIfComponentNotFoundIfComponentNotFoundifComponentNotFound is set to 'search_from_best'"search_from_best""search_from_best""search_from_best""search_from_best", such components are searched relative to the pose of the already found component from which the relative search can be performed with minimum computational effort.

The parameter PosePredictionPosePredictionPosePredictionPosePredictionposePrediction determines whether the pose of components that could not be found should be estimated. If PosePredictionPosePredictionPosePredictionPosePredictionposePrediction is set to 'none'"none""none""none""none", only the poses of the found components are returned. In contrast, if PosePredictionPosePredictionPosePredictionPosePredictionposePrediction is set to 'from_neighbors'"from_neighbors""from_neighbors""from_neighbors""from_neighbors" or 'from_all'"from_all""from_all""from_all""from_all", the poses of components that could not be found are estimated and returned with a score of ScoreCompScoreCompScoreCompScoreCompscoreComp = 0.0. The estimation of the poses is then either based on the poses of the found neighboring components in the search tree ('from_neighbors'"from_neighbors""from_neighbors""from_neighbors""from_neighbors") or on the poses of all found components ('from_all'"from_all""from_all""from_all""from_all").

Internally, the shape-based matching is used for the component-based matching in order to search the individual components (see find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModel). Therefore, the parameters MinScoreCompMinScoreCompMinScoreCompMinScoreCompminScoreComp, SubPixelCompSubPixelCompSubPixelCompSubPixelCompsubPixelComp, NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsComp, and GreedinessCompGreedinessCompGreedinessCompGreedinessCompgreedinessComp have the same meaning as the corresponding parameters in find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModel. These parameters must either contain one element, in which case the parameter is used for all components, or must contain the same number of elements as model components in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID, in which case each parameter element refers to the corresponding component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID. NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsComp may also contain two elements or twice the number of elements as model components. The first value determines the number of pyramid levels to use. The second value determines the lowest pyramid level to which the found matches are tracked. If different values should be used for different components, the number of pyramid levels and the lowest pyramid level must be specified interleaved in NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsComp. If, for example, two components are contained in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID, and the number of pyramid levels is 5 for the first component and 4 for the second component, and the lowest pyramid level is 2 for the first component and 1 for the second component, NumLevelsCompNumLevelsCompNumLevelsCompNumLevelsCompnumLevelsComp = [5,2,4,1] must be selected. Besides the subpixel extraction, SubPixelCompSubPixelCompSubPixelCompSubPixelCompsubPixelComp may also contain a second element that contains the maximum object deformation. The deformation must be specified in pixels. This can be done by passing the optional parameter value 'max_deformation '"max_deformation ""max_deformation ""max_deformation ""max_deformation " followed by an integer value between 0 and 32 (in the same string), which specifies the maximum deformation. To get a meaningful score value and to avoid erroneous matches, we recommend to always combine the allowance of a deformation with a subpixel extraction that applies a least-squares adjustment. If the subpixel extraction and/or the maximum object deformation is specified separately for each component, for each component in ComponentModelIDComponentModelIDComponentModelIDComponentModelIDcomponentModelID exactly one value for the subpixel extraction must be passed in SubPixelCompSubPixelCompSubPixelCompSubPixelCompsubPixelComp. After each value for the subpixel extraction optionally a second value can be passed, which describes the maximum object deformation of the corresponding mode. If for a certain component no value for the maximum object deformation is passed, the component is searched without taking deformations into account. Further details can be found in the documentation of find_shape_modelsfind_shape_modelsFindShapeModelsFindShapeModelsFindShapeModels.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Result

If the parameter values are correct, the operator find_component_modelfind_component_modelFindComponentModelFindComponentModelFindComponentModel returns the value 2 (H_MSG_TRUE). If the input is empty (no input image available) the behavior can be set via set_system('no_object_result',<Result>)set_system("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>). If necessary, an exception is raised.

Possible Predecessors

create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModel, create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModel, read_component_modelread_component_modelReadComponentModelReadComponentModelReadComponentModel

Possible Successors

get_found_component_modelget_found_component_modelGetFoundComponentModelGetFoundComponentModelGetFoundComponentModel

Alternatives

find_shape_modelsfind_shape_modelsFindShapeModelsFindShapeModelsFindShapeModels

See also

find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModel, find_shape_modelsfind_shape_modelsFindShapeModelsFindShapeModelsFindShapeModels, get_shape_model_paramsget_shape_model_paramsGetShapeModelParamsGetShapeModelParamsGetShapeModelParams, get_component_model_paramsget_component_model_paramsGetComponentModelParamsGetComponentModelParamsGetComponentModelParams, train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponents, set_shape_model_originset_shape_model_originSetShapeModelOriginSetShapeModelOriginSetShapeModelOrigin, smallest_rectangle2smallest_rectangle2SmallestRectangle2SmallestRectangle2SmallestRectangle2

Module

Matching