3D Matching

List of Sections ↓

This chapter gives an overview of the different 3D matching approaches available in HALCON.

3D Box Finder

As it is already contained in its name, the box finder can be used to locate box-shaped objects in 3D data. Thereby, no model of the object is needed as an input for the operator find_box_3dfind_box_3dFindBox3dFindBox3dFindBox3dfind_box_3d, but only the dimensions of the boxes to be found. As a result you can retrieve the pose of a gripping point, which can be especially useful in the case of a bin picking application.

Surface-Based Matching

The surface-based matching approach is suited to locate more complex objects as well. The shape of these objects is passed to the operator find_surface_modelfind_surface_modelFindSurfaceModelFindSurfaceModelFindSurfaceModelfind_surface_model, or find_surface_model_imagefind_surface_model_imageFindSurfaceModelImageFindSurfaceModelImageFindSurfaceModelImagefind_surface_model_image respectively, in the form of a surface model. The poses of the found object instances in the scene are then returned.

Note that there are several different approaches when using surface-based matching. For detailed explanations regarding when and how to use these approaches, tips, tricks, and troubleshooting, have a look at the technical note on Surface-Based Matching.

Deformable Surface-Based Matching

In case an object can occur in the scene in different, deformed states you can use a deformable surface model to locate the object in the scene. If an instance of such an object is found by the operator find_deformable_surface_modelfind_deformable_surface_modelFindDeformableSurfaceModelFindDeformableSurfaceModelFindDeformableSurfaceModelfind_deformable_surface_model, the object model can be retrieved featuring the respective deformation and pose.

Shape-Based Matching

With shape-based matching, instances of a 3D CAD model are searched in 2D images instead of 3D point clouds. For this, the edges of the wanted object need to be clearly visible in the image and the used camera needs to be calibrated beforehand. As a result, the object pose is computed and returned by the operator find_shape_model_3dfind_shape_model_3dFindShapeModel3dFindShapeModel3dFindShapeModel3dfind_shape_model_3d.

3D Gripping Point Detection

3D Gripping Point Detection is a deep-learning-based approach to detect gripping points on arbitrary objects in a 3D scene. For further information please see the chapter 3D Matching / 3D Gripping Point Detection.

List of Sections

• 3D Box
• 3D Gripping Point Detection
• Deformable Surface-Based
• Shape-Based
• Surface-Based