Automatic Object Detection (with SAM Meta)

The Automatic Object Segmentation module uses the Segment Anything Model (SAM) [1] developed by Meta to automatically detect and segment objects in images.

After validation, the module outputs a label image of the segmentation generated by SAM.

This module is available in the Machine Learning (AI) tools section inside the toolbar.

Automatic Object Segmentation module inside IPSDK Explorer

When clicking on the module button, a configuration window opens, allowing to adjust several segmentation parameters before launching the process.

Configuration window

Model Selection

A dropdown menu allows to select an installed SAM model or download additional ones.

SAM models parameter

Available models are:

sam vit-b (Base)
sam vit-l (Large)
sam vit-h (Huge)

The vit-b model is already installed and usable, while other models needs to be downloaded first to be available.

When clicking on the

button, a new window opens, allowing to download additional models.

SAM models download window

Model Differences

These models differ mainly in network size, accuracy, and computational cost.

Model	Accuracy	Speed	Memory Usage	Recommended Use
SAM VIT-B	Good	Fast	Low	Quick segmentation, limited hardware
SAM VIT-L	Very good	Medium	Medium	Balanced performance
SAM VIT-H	Best	Slow	High	High-quality segmentation

Note: Larger models generally produce more precise object boundaries.; Smaller models are faster and better suited for real-time or low-resource environments.

Object Size Preset

This option allows to control the expected size of objects to segment.

Object size parameter

This parameter adjusts the internal sampling strategy of SAM to focus on objects of a given scale.

Here is an example detecting small objects:

Detection of small objects

As we can see, the segmentation includes small objects to the result image.

In contrast, when focusing on large object detection, small objects are excluded from the segmentation results.

Here's, an example while detecting very large objects:

Detection of very large objects

Note

Use "Very Small" for particles or fine structures.
Use "Large" or "Very Large" for macroscopic objects or large connected regions.
Focusing on "Large" object detection significantly reduces processing time.

Threshold Accuracy

The Threshold Accuracy parameter defines the minimum confidence required for a predicted mask to be accepted.

Its value can be defined between 1 and 99.

Higher values produce fewer but more reliable segments, while lower values allow more detections but may introduce noise or uncertain regions.

Recommended Usage

Here's the recommended usage for basic processing.

Threshold	Behavior
90-80 (High)	Cleaner segmentation & Fewer false positives
80-50 (Medium))	Balanced detection
50-10 (Low)	Detection of faint or low-contrast objects

Remove Border Objects

When enabled, the remove border parameter removes all segmented labels that touch the image borders.

Remove border parameter

This option is useful when:

Border objects are incomplete,
Partial objects should be excluded from measurements.

Remove Small Shapes

When enabled, the remove small shapes parameter removes segmented objects below a minimum size.

Remove Small shapes parameter

Any segmented label with an area smaller than the minimum size value will be deleted.

Typical use cases:

Removing segmentation noise,
Eliminating tiny artifacts,
Keeping only meaningful objects.

Remove Outliers

This option identifies and removes outliers, defined as objects that significantly deviate from the majority population.

Outliers detection relies on the label analysis method and is based on feature distances between objects, such as:

Color statistics,
Shape metrics.

Remove outliers parameter

Tolerance Parameter

This parameter allows to filter the detected objects.

Tolerance	Behavior
<10 (Low)	Aggressive filtering. Only very similar objects are kept.
~10 (Medium))	Balanced detection
>10 (High)	Conservative filtering. Most objects are preserved.

Here's examples of the differences between a high and a low tolerance :

High Tolerance (15.0)	Low Tolerance (3.0)

Note: If most segmented objects share similar appearance, isolated detections with very different properties may be classified as outliers and removed.

This option is useful for:

Removing background detections,
Eliminating segmentation artifacts,
Keeping homogeneous object populations.

Output Probability Map

When enabled, the module outputs an additional image representing the probability (or confidence) map generated by SAM.

Probability map result

This map can be used to:

Visualize model confidence,
Analyze uncertain regions,
Adjust threshold parameters,
Evaluate segmentation quality.

Run Segmentation

Clicking the Validate button starts the segmentation process.

The module uses the currently selected image in the software as input.

Start of a process

Output Results

The module generates:

A labeled segmentation image (SAM result)
Optionally, a probability map (if enabled)

The results are automatically added to the current workspace.

Script generation

It is possible to generate a python script through the interface of IPSDK Explorer, by right clicking the result image and using the Generate complete script button.

Generation of a python script to use SAM models inside macros or outside the software

The following code creates a segmented label image using SAM Meta :

# Automatically selects the computing device (GPU or CPU)
device = fctSam.GetDevice(preferCuda=True)
''' Starts SAM execution on the input image inImg1.
    modelType : Version of the sam model used.
    checkpointPath : Path to the sam model.
    device : Device selected between GPU and CPU.
    pointsPerSide : Sampling density used by SAM to generate the masks.
    predIouThresh : Minimum quality threshold of the predicted labels.
    stabilityScoreThresh : Filters unstable labels.
    boxOverlapThreshold : Avoids labels shifting from the original image.
'''
imageLabel = fctSam.RunSamAutomatic2D(inImg1,
                                      modelType='vit_b',
                                      checkpointPath=os.path.join(vrb.folderSamMetaModels,'sam_vit_b_01ec64.pth'),
                                      device =device,
                                      pointsPerSide=48,
                                      predIouThresh=0.9,
                                      stabilityScoreThresh=0.9,
                                      boxOverlapThreshold=0.3
                                     )

It is possible to adjust more precisely the parameters of the RunSamAutomatic2D function (see the script comments above for parameters details).

The second part of the script is used for post processing the raw segmentation result of SAM:

# Removes objects touching the edges of the image
imageLabel = advmorpho.removeBorder2dImg(imageLabel)
# Returns a label image directly usable for future processing
output = fctSam.cleanImageLabelAfterSam(imageLabel,sizeSmallShape = 200)
# Automatically filter atypical objects based on morphological and statistical measurements
listMeasures = ['Area2dMsr','Circularity2dMsr','ConvexityMsr','MeanMsr','MedianMsr','StdDevMsr']
output = fctSam.removeOutliers(inImg1,
                               output,
                               listMeasures = listMeasures,
                               tolerance =8.0,
                               sizeSmallShape = 200
                              )

The set of measurements used for outliers detection can be modified by editing the listMeasures variable in the script.

The following illustrates the differences between the raw SAM Meta output and the results produced by IPSDK :

Input image	Native SAM result	IPSDK SAM result

[1] https://ai.meta.com/research/sam2/

Sam paper : https://arxiv.org/pdf/2304.02643