Image Processing · Pupil

What is Mathematical Morphology?

Mathematical morphology reshapes binary regions with a small geometric probe called a structuring element. Learn erosion, dilation, opening, closing, and morphological gradients, then apply them with Pupil.

Imagine an industrial arm counting metal washers on a conveyor. Color or intensity segmentation produces a binary image in which the washers are white and the conveyor is black. The result is almost correct, but reflections create isolated white specks, one washer rim contains a small black gap, and two nearby washers are joined by a thin bridge. These are errors of shape and connectivity. Mathematical morphology edits the mask using explicit geometric rules.

1. What is Mathematical Morphology?

Mathematical morphology transforms shapes by moving a small probe, called a structuring element, across an image. This tutorial explains erosion and dilation, builds opening and closing from them, and shows how to use morphological Pupil Skills without erasing important robot-perception evidence.

Morphology is easiest to understand on a binary mask. Let foreground pixels represent the selected objects and background pixels everything else. The algorithm does not know whether a foreground region is a bolt, a shadow, or dust. It asks only whether the chosen probe fits inside or touches that region.

1.1 The Morphology Pipeline

Binary shapes + probe → transformed shapes

Binary maskForeground and background pixels
Structuring elementA small shape used as a local probe
Probe every locationTranslate the structuring element across the mask
Erode or dilateRequire a full fit or any overlap
Compose operationsOpening, closing, or morphological gradient
Reshaped maskNoise removed, gaps filled, or boundaries extracted

The structuring element is a geometric question: features smaller than, thinner than, or unable to contain that probe are treated differently.

Mathematical morphology transforms a mask by testing local shape with a structuring element. Erosion and dilation are the primitives; opening and closing are ordered combinations. Adapted as an explanatory diagram fromSerra, Image Analysis and Mathematical Morphology.

Start with the washer mask. White pixels mark material believed to belong to a washer; black pixels mark the conveyor and the holes. A small second shape, called a structuring element, is moved across this binary image to test the local geometry.

Picture placing that small shape over every pixel in turn. Dilation asks whether it touches any foreground and grows the mask wherever it does. Erosion asks whether it fits completely inside the foreground and removes pixels wherever it does not. These two simple questions are enough to grow regions, shrink them, close gaps, and remove isolated artifacts.

The shape of the probe is part of the instruction. A small ellipse treats every direction similarly. A horizontal rectangle can bridge a break along a cable while leaving a vertical gap unchanged. Its size states which features count as small enough to remove or connect.

Opening and closing simply apply erosion and dilation in different orders. The diagram therefore shows a small vocabulary of geometric operations being composed into a cleaner mask. The deeper sections explain what each order preserves and what it can accidentally destroy.

1.2 Dilation: Keep Locations Where the Probe Touches Foreground

Suppose a logistics cobot segments a printed carton label, but glare breaks one character into nearby fragments. Dilation asks a generous question at every location: does the probe touch any existing white pixel? Saying yes grows the white region, which can reconnect the character—but can also join two characters that should remain separate.

To state that rule precisely, represent the foreground as a set AA of pixel coordinates and the structuring element as a set BB. Dilation is

A⊕B={z∣(B^)z∩A≠∅}.A\oplus B=\{z\mid (\hat B)_z\cap A\neq\varnothing\}.

B^\hat B is the reflection of BB about its origin, (B^)z(\hat B)_z is that reflected probe translated to location zz, and ∩\cap denotes set intersection. A location becomes foreground whenever the translated probe touches the original foreground.

The visible effect is growth. Dilation thickens regions, bridges narrow gaps, and can fill small holes. It can also merge two separate objects, so “improved connectivity” may become a counting error.

1.3 Erosion: Keep Locations Where the Probe Fits Completely

Now imagine small white dust specks in the manufacturing washer mask. Erosion asks a stricter question: does the whole probe fit inside the white region? It fits inside a large washer but not inside a tiny speck, so the speck disappears as every foreground boundary moves inward.

The set rule is

A⊖B={z∣Bz⊆A}.A\ominus B=\{z\mid B_z\subseteq A\}.

BzB_z is the probe translated to zz, and ⊆\subseteq means every point in the probe must lie inside foreground AA. Regions shrink because boundary locations fail this full-fit test.

Erosion removes small bright specks, breaks narrow bridges, and pulls boundaries inward. It can separate objects that touch through a thin connection, but it can also delete small valid parts.

Interactive mechanism

Apply a 3 × 3 probe to a binary mask

—
3 × 3 structuring element
Input mask
speck · hole · narrow bridge
Output mask

The input never changes. Comparing it with the output reveals exactly which geometry the selected operation removes, grows, opens, or fills.

1.4 Opening: Remove Small Bright Features

For the industrial arm inspecting washers, erosion removes dust but also makes every valid washer smaller. Opening follows erosion with dilation. The large washer regions survive the first step and grow back during the second, while dust specks that vanished cannot return.

Opening applies erosion and then dilation using the same probe:

A∘B=(A⊖B)⊕B.A\circ B=(A\ominus B)\oplus B.

A∘BA\circ B is the opened foreground. The erosion removes regions and protrusions that cannot contain the probe. The dilation restores the surviving regions toward their original size, but deleted structures do not return.

This is why opening is useful after thresholding: it can remove isolated bright noise and narrow bridges while approximately preserving larger components. It is not a generic “clean mask” button. If a real pin or wire is thinner than the structuring element, opening classifies it as disposable geometry.

1.5 Closing: Fill Small Dark Gaps

Imagine a humanoid segmenting a dark charging cable whose mask contains small breaks caused by reflections. Closing solves this complementary problem: it first grows the cable mask across narrow black gaps, then shrinks the outside boundary back toward its original width.

Closing reverses the opening order:

A∙B=(A⊕B)⊖B.A\bullet B=(A\oplus B)\ominus B.

A∙BA\bullet B is the closed foreground. Dilation first bridges small breaks and fills narrow holes. Erosion then pulls the expanded outer boundary back. Closing is useful for broken silhouettes and small dark gaps, but it may merge nearby parts or erase a meaningful hole.

Opening and closing are not inverses. Once an operation removes or fills a feature, applying the other operation does not reconstruct the original evidence.

1.6 Morphological Gradient, Top-Hat, and Black-Hat

An aerospace drone may need the outline of a seal, tiny bright fasteners, or small dark pits rather than a cleaned mask. Comparing the original, grown, shrunken, opened, and closed images exposes exactly these scale-dependent structures.

The morphological gradient subtracts the eroded image from the dilated image:

GB(A)=(A⊕B)−(A⊖B).G_B(A)=(A\oplus B)-(A\ominus B).

GB(A)G_B(A) is a band around the foreground boundary. Its thickness depends on the structuring element. Unlike Sobel, it derives boundaries from region geometry rather than intensity derivatives.

Top-hat subtracts the opening from the original, isolating small bright structures that opening removed. Black-hat subtracts the original from its closing, isolating small dark structures that closing filled. Both make “smaller than this probe” an explicit feature definition.

1.7 Iterations and Kernel Size Are Both Scale Controls

Choosing the operation is only half the design. We must also decide what “small” means. The probe size and the number of repetitions control that scale, and both must be tied to the features the robot may safely remove.

Pupil exposes kernel_size, kernel_shape, and iterations. A larger kernel changes the geometric probe itself. More iterations repeat the operation and compound its effect. They are related but not always equivalent, especially for different kernel shapes and composite operations.

Calibrate in pixels using deployment resolution. If the camera resolution or distance changes, a five-pixel gap represents a different physical size. For metrology, relate the kernel to millimeters through calibration or keep imaging geometry fixed.

1.8 The Structuring Element Encodes Prior Geometry

Scale alone does not describe the probe. Its shape expresses an assumption about which directions and local patterns belong together, so choosing it is a small piece of geometric modeling.

A square grows corners and axis-aligned regions differently from an ellipse. A cross connects through horizontal and vertical neighbors while treating diagonals differently. A long horizontal rectangle can close breaks in a conveyor edge without joining components separated vertically.

Its origin—the point associated with the output pixel—also matters, although symmetric odd-sized probes place it naturally at the center. Even-sized or asymmetric probes can shift boundaries while transforming them.

Choose the probe from the physical nuisance. If dust specks are roughly circular and smaller than five pixels, a small ellipse is a defensible opening element. If the task is repairing broken horizontal traces, a horizontal line expresses that intent more directly.

1.9 Duality Explains Why Erosion and Dilation Come in Pairs

The operations may feel like separate tools, but foreground and background are two views of the same binary image. Swapping them turns growth into shrinkage. This duality helps predict what an operation will do to holes as well as objects.

Dilation of the foreground is related to erosion of the background:

(A⊕B)c=Ac⊖B^.(A\oplus B)^c=A^c\ominus \hat B.

AcA^c is the complement of foreground set AA, and B^\hat B is the reflected structuring element. This duality explains why growing bright objects also shrinks dark holes, and why eroding bright objects expands the surrounding background.

Opening and closing are dual in the same way. If a desired defect is dark on a bright part, reasoning about the background can clarify whether closing or a black-hat transform is appropriate.

1.10 Grayscale Morphology Extends the Same Idea

So far the probe has asked whether binary pixels are on or off. The same geometric idea can operate directly on intensity images by replacing “any foreground” with a local maximum and “all foreground” with a local minimum.

For a flat structuring element, grayscale dilation takes the local maximum and grayscale erosion takes the local minimum:

(I⊕B)(x,y)=max⁡(u,v)∈BI(x−u,y−v),(I\oplus B)(x,y)=\max_{(u,v)\in B}I(x-u,y-v), (I⊖B)(x,y)=min⁡(u,v)∈BI(x+u,y+v).(I\ominus B)(x,y)=\min_{(u,v)\in B}I(x+u,y+v).

II is a grayscale image and BB is the neighborhood. Dilation spreads bright peaks; erosion spreads dark valleys. Grayscale opening estimates a background that excludes bright features smaller than the probe. Subtracting it from the original yields the top-hat response. Closing estimates over small dark features; subtracting the original yields black-hat.

This is useful for uneven illumination. A top-hat transform can isolate small bright fasteners against a slowly changing background without selecting one global brightness threshold.

1.11 Thinning Preserves Connectivity While Reducing Width

A humanoid routing a cable cares about the cable’s path and endpoints more than its filled pixel area. Reducing the cable mask to a narrow centerline makes length, bends, branches, and endpoints easier to measure.

Thinning, or skeletonization, repeatedly removes boundary pixels under rules designed to preserve topology. A thick cable mask becomes a one-pixel-wide centerline. The result supports length measurement, branch detection, endpoints, and path planning through elongated shapes.

Skeletons are sensitive to boundary noise: every small bump can create a branch. Clean the mask carefully before thinning, but avoid closing gaps that represent real separation. The skeleton is a topological summary, not a metric replacement for the original region width.

1.12 Hit-or-Miss Searches for a Specific Local Pattern

Growth and shrinkage treat many local arrangements alike. If the task is to find one exact binary arrangement—such as an endpoint or junction—we need an operation that tests a foreground pattern and its surrounding background together.

The hit-or-miss transform performs that test. It can locate line endpoints, junctions, or small binary motifs.

This is more explicit than learned recognition: the structuring elements define the exact local configuration. It is useful when imaging geometry is controlled and the target pattern is stable, but brittle under rotation, scale change, or segmentation noise unless several pattern variants are tested.

1.13 Morphology Can Change Topology and Measurements

The visual result may look cleaner, but cleanup can change the facts a robot later measures. Before using morphology in production, we must check not only removed noise but also component count, holes, boundary location, and object identity.

Closing can turn two components into one. Opening can split a thinly connected component into two. Filling a hole changes Euler topology, area, centroid, contour hierarchy, and possibly the object’s inferred identity.

That may be the intended correction, but it should never be invisible. Log component count, hole count, area, and centroid before and after cleanup. If the robot uses contour geometry for grasping or inspection, quantify the boundary displacement introduced by the operation.

A helpful rule is to tune morphology against the smallest valid feature and the largest invalid artifact simultaneously. If their pixel scales overlap, morphology alone cannot separate them reliably; improve lighting, optics, segmentation, or add semantic context.

2. How to Use Morphology in the Telekinesis Agentic OS

Telekinesis provides separate Pupil Skills for each operation. The example below removes small bright noise with opening and then fills narrow gaps with closing.

from telekinesis import datatypes, pupil

mask = datatypes.Image.from_url(
    url="https://assets.telekinesis.ai/examples/v1/images/broken_cables.webp"
).to_grayscale()

opened = pupil.filter_image_using_morphological_open(
    image=mask,
    kernel_size=3,
    kernel_shape="ellipse",
    iterations=1,
    border_type="constant",
    border_value=0,
)

closed = pupil.filter_image_using_morphological_close(
    image=opened,
    kernel_size=5,
    kernel_shape="ellipse",
    iterations=1,
    border_type="constant",
    border_value=0,
)

print(closed.shape)

Inspect the intermediate opened mask. If only the final output is viewed, a disappearing feature or merged pair of objects can be incorrectly attributed to the wrong stage.

Try it out

Clean binary masks with Pupil

See the morphological opening interface, kernel shapes, parameter ranges, and example pipeline.

Open morphology docs

Runnable example

Run the morphological opening example

Open the exact Telekinesis Python example for binary mask cleanup.

View Python example

3. How to Evaluate Morphological Cleanup

Morphology should be evaluated as a geometric edit, not a cosmetic improvement. The useful question is whether it removed the intended artifact without changing valid object structure.

Measure more than mask appearance. Count connected components, compare hole counts, record area and centroid changes, and check the minimum feature width that survives. For a segmentation ground truth, report intersection over union, but also evaluate the property used by the robot: part count, grasp-region stability, defect recall, or dimensional error.

Include negative cases. Opening may delete a small true object; closing may join two objects that should remain separate. A useful validation set therefore contains both the artifacts the operation should fix and similarly sized structures it must preserve.

4. Where to Go Next

5. References