Image & Point Cloud Fundamentals · Fundamentals

What is a Binary Mask?

A binary mask marks each image pixel as selected or not selected. Learn how masks isolate usable regions, how they differ from photographs, and how Telekinesis represents them for robot perception.

Imagine an industrial arm preparing to pick one component from a tray. A color image shows the entire scene, but the arm may only need the pixels belonging to the chosen component. A binary mask provides that selection.

1. What is a Binary Mask?

A binary mask is a two-dimensional grid that labels every image position as either selected or not selected. It normally has the same height and width as the image it describes, so every mask pixel corresponds to one image pixel.

An image and its corresponding binary mask

Write a mask as

M(u,v)∈{0,1}.M(u,v)\in\{0,1\}.

MM is the mask, (u,v)(u,v) is an image position, 1 means the pixel is selected, and 0 means it is not. In a foreground mask, selected pixels belong to the foreground object. The same mechanism can instead mark a safety zone, a region to inspect, or pixels with valid depth.

1.1 A Mask Stores a Decision, Not Appearance

A photograph may store three color values at every location. A binary mask stores only the result of a decision. It does not remember whether the selected part was blue, textured, bright, or scratched.

This makes masks useful as a bridge between perception stages. A segmentation model can create a mask, morphology can remove small gaps, contour detection can trace its boundary, and a depth pipeline can keep only 3D points whose image pixels are selected.

Despite the name, a binary mask is not always packed as one physical bit per pixel. It is commonly stored in an integer array—often uint8—whose values are restricted to 0 and 1.

1.2 Binary, Semantic, and Instance Masks

These related representations answer different questions:

RepresentationPixel value meansExample
Binary maskSelected or not selected“Pixels belonging to any cable”
Semantic maskA category ID“Cable, connector, workbench”
Instance maskA particular object ID“Cable 1, cable 2, connector 1”

A semantic or instance mask can be converted into a binary mask by keeping the labels of interest and setting everything else to zero. Information is lost during that conversion: after two object instances are merged into 1, the binary mask no longer says which pixel came from which object.

1.3 What Can Go Wrong?

A mask can look clean and still represent the wrong region. Shadows may join two components, glare may punch holes into a part, and a thin cable may disappear at low resolution. Before an industrial arm moves, check whether the selected region is large enough, connected as expected, and consistent with depth and workspace constraints.

2. Binary Masks in Telekinesis

Telekinesis represents label images with datatypes.SegmentationImage. It stores a two-dimensional integer array. For a binary mask, use label 0 for the background and 1 for the selected region.

import numpy as np
from telekinesis import datatypes

labels = np.zeros((480, 640), dtype=np.uint8)
labels[120:300, 220:410] = 1

mask = datatypes.SegmentationImage(labels)
print(mask.label_codes)       # [0 1]
print(mask.number_of_labels)  # 2

SegmentationImage can also contain several non-negative label codes. Its to_binary() method keeps zero as background and maps every nonzero label to one, which is convenient when the downstream step only needs foreground versus background.

3. Where to Go Next