train_model_componentsT_train_model_componentsTrainModelComponentsTrainModelComponentstrain_model_components (Operator)

Name

train_model_componentsT_train_model_componentsTrainModelComponentsTrainModelComponentstrain_model_components — Train components and relations for the component-based matching.

Signature

Description

train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components extracts the final (rigid) model components and trains their mutual relations, i.e., their relative movements, on the basis of the initial components by considering several training images. The result of the training is returned in the handle ComponentTrainingIDComponentTrainingIDComponentTrainingIDComponentTrainingIDcomponentTrainingIDcomponent_training_id. The training result can be subsequently used to create the actual component model using create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model.

train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components should be used in cases where the relations of the components are not known and should be trained automatically. In contrast, if the relations are known no training needs to be performed with train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components. Instead, the component model can be directly created with create_component_modelcreate_component_modelCreateComponentModelCreateComponentModelCreateComponentModelcreate_component_model.

If the initial components have been automatically created by using gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components, InitialComponentsInitialComponentsInitialComponentsInitialComponentsinitialComponentsinitial_components contains the contour regions of the initial components. In contrast, if the initial components should be defined by the user, they can be directly passed in InitialComponentsInitialComponentsInitialComponentsInitialComponentsinitialComponentsinitial_components. However, instead of the contour regions for each initial component, its enclosing region must be passed in the tuple. The (contour) regions refer to the model image ModelImageModelImageModelImageModelImagemodelImagemodel_image. If the initial components have been obtained using gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components, the model image should be the same as in gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components. Please note that each initial component is part of at most one rigid model component. This is because during the training initial components can be merged into rigid model components if required (see below). However, they cannot be split and distributed to several rigid model components.

train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components uses the following approach to perform the training: In the first step, the initial components are searched in all training images. In some cases, one initial component may be found in an training image more than once. Thus, in the second step, the resulting ambiguities are solved, i.e., the most probable pose of each initial component is found. Consequently, after solving the ambiguities, in all training images at most one pose of each initial component is available. In the next step the poses are analyzed and those initial components that do not show any relative movement are clustered to the final rigid model components. Finally, in the last step the relations between the model components are computed by analyzing their relative poses over the sequence of training images. The parameters that are associated with the mentioned steps are explained in the following.

The training is performed based on several training images, which are passed in TrainingImagesTrainingImagesTrainingImagesTrainingImagestrainingImagestraining_images. Each training image must show at most one instance of the compound object and should contain the full range of allowed relative movements of the model components. If, for example, the component model of an on/off switch should be trained, one training image that shows the switch turned off is sufficient if the switch in the model image is turned on, or vice versa.

The principle of the training is to find the initial components in all training images and to analyze their poses. For this, for each initial component a shape model is created (see create_shape_modelcreate_shape_modelCreateShapeModelCreateShapeModelCreateShapeModelcreate_shape_model), which is then used to determine the poses (position and orientation) of the initial components in the training images (see find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model). Depending on the mode that is set by using set_system('pregenerate_shape_models',...)set_system("pregenerate_shape_models",...)SetSystem("pregenerate_shape_models",...)SetSystem("pregenerate_shape_models",...)SetSystem("pregenerate_shape_models",...)set_system("pregenerate_shape_models",...), the shape model is either pregenerated completely or computed online during the search. The mode influences the computation time as well as the robustness of the training. Furthermore, it should be noted that if single-channel image are used in ModelImageModelImageModelImageModelImagemodelImagemodel_image as well as in TrainingImagesTrainingImagesTrainingImagesTrainingImagestrainingImagestraining_images the metric 'use_polarity'"use_polarity""use_polarity""use_polarity""use_polarity""use_polarity" is used internally for create_shape_modelcreate_shape_modelCreateShapeModelCreateShapeModelCreateShapeModelcreate_shape_model, while if multichannel images are used in either ModelImageModelImageModelImageModelImagemodelImagemodel_image or TrainingImagesTrainingImagesTrainingImagesTrainingImagestrainingImagestraining_images the metric 'ignore_color_polarity'"ignore_color_polarity""ignore_color_polarity""ignore_color_polarity""ignore_color_polarity""ignore_color_polarity" is used. Finally, it should be noted that while the number of channels in ModelImageModelImageModelImageModelImagemodelImagemodel_image and TrainingImagesTrainingImagesTrainingImagesTrainingImagestrainingImagestraining_images may be different, e.g., to facilitate model generation from synthetically generated images, the number of channels in all the images in TrainingImagesTrainingImagesTrainingImagesTrainingImagestrainingImagestraining_images must be identical. For further details see create_shape_modelcreate_shape_modelCreateShapeModelCreateShapeModelCreateShapeModelcreate_shape_model. The creation of the shape models can be influenced by choosing appropriate values for the parameters ContrastLowContrastLowContrastLowContrastLowcontrastLowcontrast_low, ContrastHighContrastHighContrastHighContrastHighcontrastHighcontrast_high, and MinSizeMinSizeMinSizeMinSizeminSizemin_size. These parameters have the same meaning as in gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components and can be automatically determined by passing 'auto'"auto""auto""auto""auto""auto": If both hysteresis threshold should be automatically determined, both ContrastLowContrastLowContrastLowContrastLowcontrastLowcontrast_low and ContrastHighContrastHighContrastHighContrastHighcontrastHighcontrast_high must be set to 'auto'"auto""auto""auto""auto""auto". In contrast, if only one threshold value should be determined, ContrastLowContrastLowContrastLowContrastLowcontrastLowcontrast_low must be set to 'auto'"auto""auto""auto""auto""auto" while ContrastHighContrastHighContrastHighContrastHighcontrastHighcontrast_high must be set to an arbitrary value different from 'auto'"auto""auto""auto""auto""auto". If the initial components have been automatically created by gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components, the parameters ContrastLowContrastLowContrastLowContrastLowcontrastLowcontrast_low, ContrastHighContrastHighContrastHighContrastHighcontrastHighcontrast_high, and MinSizeMinSizeMinSizeMinSizeminSizemin_size should be set to the same values as in gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components.

To influence the search for the initial components, the parameters MinScoreMinScoreMinScoreMinScoreminScoremin_score, SearchRowTolSearchRowTolSearchRowTolSearchRowTolsearchRowTolsearch_row_tol, SearchColumnTolSearchColumnTolSearchColumnTolSearchColumnTolsearchColumnTolsearch_column_tol, SearchAngleTolSearchAngleTolSearchAngleTolSearchAngleTolsearchAngleTolsearch_angle_tol, and TrainingEmphasisTrainingEmphasisTrainingEmphasisTrainingEmphasistrainingEmphasistraining_emphasis can be set. The parameter MinScoreMinScoreMinScoreMinScoreminScoremin_score determines what score a potential match must at least have to be regarded as an instance of the initial component in the training image. The larger MinScoreMinScoreMinScoreMinScoreminScoremin_score is chosen, the faster the training is. If the initial components can be expected never to be occluded in the training images, MinScoreMinScoreMinScoreMinScoreminScoremin_score may be set as high as 0.8 or even 0.9 (see find_shape_modelfind_shape_modelFindShapeModelFindShapeModelFindShapeModelfind_shape_model).

By default, the components are searched only at points in which the component lies completely within the respective training image. This means that a component will not be found if it extends beyond the borders of the image, even if it would achieve a score greater than MinScoreMinScoreMinScoreMinScoreminScoremin_score. This behavior can be changed with set_system('border_shape_models','true')set_system("border_shape_models","true")SetSystem("border_shape_models","true")SetSystem("border_shape_models","true")SetSystem("border_shape_models","true")set_system("border_shape_models","true"), which will cause components that extend beyond the image border to be found if they achieve a score greater than MinScoreMinScoreMinScoreMinScoreminScoremin_score. Here, points lying outside the image are regarded as being occluded, i.e., they lower the score. It should be noted that the runtime of the training will increase in this mode.

When dealing with a high number of initial components and many training images, the training may take a long time (up to several minutes). In order to speed up the training it is possible to restrict the search space for the single initial components in the training images. For this, the poses of the initial components in the model image are used as reference pose. The parameters SearchRowTolSearchRowTolSearchRowTolSearchRowTolsearchRowTolsearch_row_tol and SearchColumnTolSearchColumnTolSearchColumnTolSearchColumnTolsearchColumnTolsearch_column_tol specify the position tolerance region relative to the reference position in which the search is performed. Assume, for example, that the position of an initial component in the model image is (100,200) and SearchRowTolSearchRowTolSearchRowTolSearchRowTolsearchRowTolsearch_row_tol is set to 20 and SearchColumnTolSearchColumnTolSearchColumnTolSearchColumnTolsearchColumnTolsearch_column_tol is set to 10. Then, this initial component is searched in the training images only within the axis-aligned rectangle that is determined by the upper left corner (80,190) and the lower right corner (120,210). The same holds for the orientation angle range, which can be restricted by specifying the angle tolerance SearchAngleTolSearchAngleTolSearchAngleTolSearchAngleTolsearchAngleTolsearch_angle_tol to the angle range of [-SearchAngleTolSearchAngleTolSearchAngleTolSearchAngleTolsearchAngleTolsearch_angle_tol,+SearchAngleTolSearchAngleTolSearchAngleTolSearchAngleTolsearchAngleTolsearch_angle_tol]. Thus, it is possible to considerably reduce the computational effort during the training by an adequate acquisition of the training images. If one of the three parameters is set to -1, no restriction of the search space is applied in the corresponding dimension.

The input parameters ContrastLowContrastLowContrastLowContrastLowcontrastLowcontrast_low, ContrastHighContrastHighContrastHighContrastHighcontrastHighcontrast_high, MinSizeMinSizeMinSizeMinSizeminSizemin_size, MinScoreMinScoreMinScoreMinScoreminScoremin_score, SearchRowTolSearchRowTolSearchRowTolSearchRowTolsearchRowTolsearch_row_tol, SearchColumnTolSearchColumnTolSearchColumnTolSearchColumnTolsearchColumnTolsearch_column_tol, and SearchAngleTolSearchAngleTolSearchAngleTolSearchAngleTolsearchAngleTolsearch_angle_tol must either contain one element, in which case the parameter is used for all initial components, or must contain the same number of elements as the initial components contained in InitialComponentsInitialComponentsInitialComponentsInitialComponentsinitialComponentsinitial_components, in which case each parameter element refers to the corresponding initial component in InitialComponentsInitialComponentsInitialComponentsInitialComponentsinitialComponentsinitial_components.

The parameter TrainingEmphasisTrainingEmphasisTrainingEmphasisTrainingEmphasistrainingEmphasistraining_emphasis offers another possibility to influence the computation time of the training and to simultaneously affect its robustness. If TrainingEmphasisTrainingEmphasisTrainingEmphasisTrainingEmphasistrainingEmphasistraining_emphasis is set to 'speed'"speed""speed""speed""speed""speed", on the one hand the training is comparatively fast, on the other hand it may happen in some cases that some initial components are not found in the training images or are found at a wrong pose. Consequently, this would lead to an incorrect computation of the rigid model components and their relations. The poses of the found initial components in the individual training images can be examined by using get_training_componentsget_training_componentsGetTrainingComponentsGetTrainingComponentsGetTrainingComponentsget_training_components. If erroneous matches occur the training should be restarted with TrainingEmphasisTrainingEmphasisTrainingEmphasisTrainingEmphasistrainingEmphasistraining_emphasis set to 'reliability'"reliability""reliability""reliability""reliability""reliability". This results in a higher robustness at the cost of a longer computation time.

Furthermore, during the pose determination of the initial components ambiguities may occur if the initial components are rotationally symmetric or if several initial components are identical or at least similar to each other. To solve the ambiguities, the most probable pose is calculated for each initial component in each training image. For this, the individual ambiguous poses are evaluated. The pose of an initial component receives a good evaluation if the relative pose of the initial component with respect to the other initial components is similar to the corresponding relative pose in the model image. The method to evaluate this similarity can be chosen with AmbiguityCriterionAmbiguityCriterionAmbiguityCriterionAmbiguityCriterionambiguityCriterionambiguity_criterion. In almost all cases the best results are obtained with 'rigidity'"rigidity""rigidity""rigidity""rigidity""rigidity", which assumes the rigidity of the compound object. The more the rigidity of the compound object is violated by the pose of the initial component, the worse its evaluation is. In the case of 'distance'"distance""distance""distance""distance""distance", only the distance between the initial components is considered during the evaluation. Hence, the pose of the initial component receives a good evaluation if its distances to the other initial components is similar to the corresponding distances in the model image. Accordingly, when choosing 'orientation'"orientation""orientation""orientation""orientation""orientation", only the relative orientation is considered during the evaluation. Finally, the simultaneous consideration of distance and orientation can be achieved by choosing 'distance_orientation'"distance_orientation""distance_orientation""distance_orientation""distance_orientation""distance_orientation". In contrast to 'rigidity'"rigidity""rigidity""rigidity""rigidity""rigidity", the relative pose of the initial components is not considered when using 'distance_orientation'"distance_orientation""distance_orientation""distance_orientation""distance_orientation""distance_orientation".

The process of solving the ambiguities can be further influenced by the parameter MaxContourOverlapMaxContourOverlapMaxContourOverlapMaxContourOverlapmaxContourOverlapmax_contour_overlap. This parameter describes the extent by which the contours of two initial component matches may overlap each other. Let the letters 'I' and 'T', for example, be two initial components that should be searched in a training image that shows the string 'IT'. Then, the initial component 'T' should be found at its correct pose. In contrast, the initial component 'I' will be found at its correct pose ('I') but also at the pose of the 'T' because of the similarity of the two components. To discard the wrong match of the initial component 'I', an appropriate value for MaxContourOverlapMaxContourOverlapMaxContourOverlapMaxContourOverlapmaxContourOverlapmax_contour_overlap can be chosen: If overlapping matches should be tolerated, MaxContourOverlapMaxContourOverlapMaxContourOverlapMaxContourOverlapmaxContourOverlapmax_contour_overlap should be set to 1. If overlapping matches should be completely avoided, MaxContourOverlapMaxContourOverlapMaxContourOverlapMaxContourOverlapmaxContourOverlapmax_contour_overlap should be set to 0. By choosing a value between 0 and 1, the maximum percentage of overlapping contour pixels can be adjusted.

The decision which initial components can be clustered to rigid model components is made based on the poses of the initial components in the model image and in the training images. Two initial components are merged if they do not show any relative movement over all images. Assume that in the case of the above mentioned switch the training image would show the same switch state as the model image, the algorithm would merge the respective initial components because it assumes that the entire switch is one rigid model component. The extent by which initial components are merged can be influenced with the parameter ClusterThresholdClusterThresholdClusterThresholdClusterThresholdclusterThresholdcluster_threshold. This cluster threshold is based on the probability that two initial components belong to the same rigid model component. Thus, ClusterThresholdClusterThresholdClusterThresholdClusterThresholdclusterThresholdcluster_threshold describes the minimum probability which two initial components must have in order to be merged. Since the threshold is based on a probability value, it must lie in the interval between 0 and 1. The greater the threshold is chosen, the smaller the number of initial components that are merged. If a threshold of 0 is chosen, all initial components are merged into one rigid component, while for a threshold of 1 no merging is performed and each initial component is adopted as one rigid model component.

The final rigid model components are returned in ModelComponentsModelComponentsModelComponentsModelComponentsmodelComponentsmodel_components. Later, the index of a component region in ModelComponentsModelComponentsModelComponentsModelComponentsmodelComponentsmodel_components is used to denote the model component. The poses of the components in the training images can be examined by using get_training_componentsget_training_componentsGetTrainingComponentsGetTrainingComponentsGetTrainingComponentsget_training_components.

After the determination of the model components their relative movements are analyzed by determining the movement of one component with respect to a second component for each pair of components. For this, the components are referred to their reference points. The reference point of a component is the center of gravity of its contour region, which is returned in ModelComponentsModelComponentsModelComponentsModelComponentsmodelComponentsmodel_components. It can be calculated by calling area_centerarea_centerAreaCenterAreaCenterAreaCenterarea_center. Finally, the relative movement is represented by the smallest enclosing rectangle of arbitrary orientation of the reference point movement and by the smallest enclosing angle interval of the relative orientation of the second component over all images. The determined relations can be inspected by using get_component_relationsget_component_relationsGetComponentRelationsGetComponentRelationsGetComponentRelationsget_component_relations.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.

Parameters

Example (HDevelop)

* Get the model image.
read_image (ModelImage, 'model_image.tif')
* Define the regions for the initial components.
gen_rectangle2 (InitialComponentRegions, 212, 233, 0.62, 167, 29)
gen_rectangle2 (Rectangle2, 298, 363, 1.17, 162, 34)
gen_rectangle2 (Rectangle3, 63, 444, -0.26, 50, 27)
gen_rectangle2 (Rectangle4, 120, 473, 0, 33, 20)
concat_obj (InitialComponentRegions, Rectangle2, InitialComponentRegions)
concat_obj (InitialComponentRegions, Rectangle3, InitialComponentRegions)
concat_obj (InitialComponentRegions, Rectangle4, InitialComponentRegions)
* Get the training images.
gen_empty_obj (TrainingImages)
for i := 1 to 4 by 1
    read_image (TrainingImage, 'training_image-'+i+'.tif')
    concat_obj (TrainingImages, TrainingImage, TrainingImages)
endfor
* Extract the model components and train the relations.
train_model_components (ModelImage, InitialComponentRegions, \
                        TrainingImages, ModelComponents, 22, 60, 30, 0.6, \
                        0, 0, rad(60), 'speed', 'rigidity', 0.2, 0.4, \
                        ComponentTrainingID)

Result

If the parameter values are correct, the operator train_model_componentstrain_model_componentsTrainModelComponentsTrainModelComponentsTrainModelComponentstrain_model_components returns the value TRUE. If the input is empty (no input images are available) the behavior can be set via set_system('no_object_result',<Result>)set_system("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)set_system("no_object_result",<Result>). If necessary, an exception is raised.

Possible Predecessors

gen_initial_componentsgen_initial_componentsGenInitialComponentsGenInitialComponentsGenInitialComponentsgen_initial_components

Possible Successors

inspect_clustered_componentsinspect_clustered_componentsInspectClusteredComponentsInspectClusteredComponentsInspectClusteredComponentsinspect_clustered_components, cluster_model_componentscluster_model_componentsClusterModelComponentsClusterModelComponentsClusterModelComponentscluster_model_components, modify_component_relationsmodify_component_relationsModifyComponentRelationsModifyComponentRelationsModifyComponentRelationsmodify_component_relations, write_training_componentswrite_training_componentsWriteTrainingComponentsWriteTrainingComponentsWriteTrainingComponentswrite_training_components, get_training_componentsget_training_componentsGetTrainingComponentsGetTrainingComponentsGetTrainingComponentsget_training_components, get_component_relationsget_component_relationsGetComponentRelationsGetComponentRelationsGetComponentRelationsget_component_relations, create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model, clear_training_componentsclear_training_componentsClearTrainingComponentsClearTrainingComponentsClearTrainingComponentsclear_training_components

See also

create_trained_component_modelcreate_trained_component_modelCreateTrainedComponentModelCreateTrainedComponentModelCreateTrainedComponentModelcreate_trained_component_model

Module

Matching