Image Segmentation · Cornea

What is Image Thresholding?

Image thresholding solves the problem of separating pixels with a known intensity cutoff. Learn the comparison modes, histogram reasoning, failure cases, evaluation metrics, and how to use global thresholding in the Telekinesis Agentic OS.

Imagine a camera above a backlit conveyor. The parts appear dark and the illuminated belt appears bright. If that contrast remains stable, the robot does not need a learned model to separate them. It needs one understandable rule: pixels on one side of an intensity cutoff belong to the parts; pixels on the other side belong to the background.

1. What is Image Thresholding?

Global image thresholding is a deterministic segmentation algorithm that compares every pixel with one fixed intensity cutoff. It solves foreground–background problems in which the two regions occupy reliably different intensity ranges.

“Global” means the same threshold is used everywhere. A pixel in the top-left corner and a pixel in the bottom-right corner face exactly the same decision. That simplicity makes the result fast, reproducible, and easy to audit. It also explains the main weakness: a shadow or lighting gradient can move otherwise identical surfaces to opposite sides of the threshold.

1.1 The Complete Thresholding Pipeline

One intensity cutoff → one label per pixel

Intensity imageOne value at every pixel
Manual thresholdKnown cutoff and comparison mode
Pixel comparisonCompare every intensity with the same cutoff
Output mappingBinary, inverse, truncate, or preserve
Thresholded imageForeground/background or clipped intensities

A global threshold is predictable because the rule never changes, but that same rigidity makes it sensitive to illumination gradients.

Manual global thresholding applies one comparison rule independently at every pixel; unlike Otsu or Yen, it does not estimate the cutoff from the current image. Adapted as an explanatory diagram fromSezgin and Sankur, thresholding survey.

Begin with a grayscale image in which every pixel stores one brightness value. The engineer chooses a cutoff, or threshold, that separates the values believed to belong to the foreground from those believed to belong to the background.

The algorithm then visits one pixel at a time. If its intensity falls on the selected side of the cutoff, it becomes foreground; otherwise it becomes background. Repeating that comparison across the image produces a binary mask that later code can count, measure, or use as a region of interest.

Nothing in this process understands objects. A bright reflection and a bright part receive the same label if their values cross the same threshold. Two equally dark pixels receive the same result even when they sit on opposite sides of the image. Thresholding works when intensity itself carries the answer and the lighting makes that answer separable.

The next sections first explain how a color image becomes a usable intensity and then show the exact decision made at each pixel.

1.2 From Color to One Intensity Value

Thresholding is easiest to understand for a grayscale image I(x,y)I(x,y), where (x,y)(x,y) identifies a pixel and II is its intensity. For an eight-bit image, intensities usually lie between 0 and 255.

If the source is RGB, the implementation must first reduce or otherwise interpret its channels. A common luma-like conversion is

Y′(x,y)=0.299R′(x,y)+0.587G′(x,y)+0.114B′(x,y).Y'(x,y)=0.299R'(x,y)+0.587G'(x,y)+0.114B'(x,y).

R′R', G′G', and B′B' are gamma-encoded color channels, and Y′Y' is a brightness-like encoded value. The coefficients are not a universal law of cameras; they belong to a particular video/color convention. When precise behavior matters, convert to grayscale explicitly and inspect the actual values entering the threshold operation.

1.3 Binary Thresholding as a Decision Function

The basic binary rule is

M(x,y)={1,I(x,y)>T,0,I(x,y)≤T.M(x,y)= \begin{cases} 1,&I(x,y)>T,\\ 0,&I(x,y)\le T. \end{cases}

TT is the global threshold, I(x,y)I(x,y) is the input intensity, and M(x,y)M(x,y) is the resulting label. Every pixel above TT becomes foreground. Binary inverse thresholding swaps the labels, which is useful when dark parts sit on a bright background.

The boundary convention—whether equality belongs above or below—must remain consistent. A one-level difference can affect many pixels when a large flat region lies exactly at the cutoff.

1.4 The Five Common Threshold Modes

The Telekinesis Skill exposes the familiar modes binary, binary_inv, trunc, tozero, and tozero_inv. Let TT be min_value, let AA be max_value, and let z=I(x,y)z=I(x,y).

ModeOutput ruleWhat it preserves
binaryAA if z>Tz>T, otherwise 00Only the comparison result
binary_inv00 if z>Tz>T, otherwise AAInverted comparison result
truncTT if z>Tz>T, otherwise zzDark values; clips bright values
tozerozz if z>Tz>T, otherwise 00Original bright intensities
tozero_inv00 if z>Tz>T, otherwise zzOriginal dark intensities

Only the binary modes directly produce a two-valued image. The other modes are intensity transforms that can prepare an image for later processing. The parameter name max_value can therefore be misleading: it is the replacement value used by binary modes, not a second upper bound defining an accepted interval.

1.5 How to Choose the Threshold From a Histogram

An intensity histogram counts how many pixels have each gray level. If dark parts and a bright background form two distinct populations, the histogram often contains two peaks separated by a valley. Placing TT near that valley rejects many ambiguous pixels.

This is a calibration exercise. Capture representative images from the deployed camera, including normal brightness drift. Label target and background pixels, then inspect their distributions. A useful threshold leaves margin between the brightest valid dark pixel and the darkest valid bright pixel.

Do not tune from one attractive frame. Camera exposure, gain, white balance, surface reflectivity, and lamp aging all move the histogram. A threshold selected from only the nominal frame may fail during the first real production shift.

Interactive threshold

Move the cutoff and watch the decision change

Threshold —
Intensity histogram
0 · black255 · white
Resulting one-dimensional mask

1.6 Why One Threshold Can Fail Across an Image

Suppose the left side of a conveyor receives more light than the right. The same gray part may produce intensity 80 on the left and 42 on the right. A threshold of 60 splits one physical material into foreground and background.

This failure is spatial, but a global threshold deliberately ignores position. Local or adaptive thresholding addresses it by calculating a neighborhood-dependent cutoff. Alternatively, the more reliable engineering fix may be diffuse illumination, locked exposure, or a light shield. Better optics and lighting often outperform extra software complexity.

1.7 Thresholding Has No Training Loss

Manual thresholding contains no learned parameters and uses no training dataset, so it has no loss function. The threshold is an engineering configuration value.

That does not remove the need for evaluation. For predicted foreground MM and annotated foreground GG, intersection over union is

IoU(M,G)=∣M∩G∣∣M∪G∣.IoU(M,G)=\frac{|M\cap G|}{|M\cup G|}.

M∩GM\cap G contains correctly selected foreground pixels, and M∪GM\cup G contains every pixel selected by either mask. Also report false-negative rate when missing a part is costly and false-positive rate when background pixels can trigger an unsafe pick.

1.8 From Pixel Labels to Robot Decisions

A binary mask becomes useful after spatial reasoning. Connected-component labeling groups neighboring foreground pixels into candidate parts. Area filtering rejects dust. Contours provide shape and perimeter. Moments provide centroids, and calibrated camera geometry maps those image coordinates into the robot workspace.

Thresholding itself supplies none of those guarantees. A bright reflection may form a valid foreground component. Two touching parts may merge. One part split by shadow may become two components. Validate the complete pipeline against the physical task.

1.9 When Should You Use a Manual Threshold?

SituationRecommendation
Fixed camera, stable backlight, known materialsManual threshold is a strong baseline
Threshold changes between otherwise similar framesTry Otsu or Yen and investigate exposure drift
Illumination varies across one frameUse local/adaptive thresholding or improve lighting
Target is defined by color rather than intensityUse RGB, HSV, LAB, or YCrCb segmentation
Target is selected from a known interior pointFlood fill may express the task better

Manual thresholding is often the right industrial answer precisely because it is boring. When the workcell controls the image formation, a fixed rule is easy to validate and monitor. Automatic methods are valuable when the cutoff genuinely must adapt, not merely because they sound more sophisticated.

1.10 Calibrate a Threshold Without Guessing

Start with images, not a slider. Capture frames from the darkest and brightest valid operating conditions and annotate representative target and background regions. For every annotated pixel, record its grayscale intensity. You now have two empirical distributions rather than one visually chosen example.

If the distributions do not overlap, any threshold in the gap separates the sampled data. Choose a value with margin on both sides rather than placing it against one distribution. If they overlap, no single global threshold can classify every sampled pixel correctly. The overlap is evidence to improve lighting, restrict the region of interest, use another image feature, or accept a measured error rate.

Sweep candidate thresholds across the calibration set. For each value, calculate true positives, false positives, true negatives, and false negatives. Precision and recall are

Precision⁡=TPTP+FP,Recall⁡=TPTP+FN.\operatorname{Precision}=\frac{TP}{TP+FP}, \qquad \operatorname{Recall}=\frac{TP}{TP+FN}.

TPTP counts correctly selected target pixels, FPFP counts background pixels incorrectly selected, and FNFN counts missed target pixels. Choose the operating point according to physical cost. A safety mask may prioritize recall; a pick proposal may prioritize precision to avoid grasping the background.

Freeze the chosen threshold with the camera configuration that produced it. Exposure, gain, gamma, white balance, bit depth, and grayscale conversion belong in the same versioned calibration record.

1.11 Noise, Blur, and Bit Depth Change the Boundary

Pixels close to the threshold can flip labels from frame to frame because of sensor noise. If I(x,y)=T±1I(x,y)=T\pm1, even a tiny variation changes the binary decision. A small denoising filter can stabilize flat regions, but blur also moves edges and can erase small defects. Evaluate preprocessing as part of the segmentation algorithm, not as a harmless cosmetic step.

Bit depth matters too. A threshold of 45 has different meaning in an eight-bit image, a normalized floating-point image, and a twelve-bit camera stream. Confirm the datatype and numeric range at the Skill boundary. Converting twelve-bit measurements to eight bits compresses 4096 possible values into 256 levels and can remove the intensity separation the inspection depended on.

Compression creates another source of variation. JPEG ringing around a sharp part boundary produces intensities not present on either physical surface. Lossless acquisition is preferable for metrology and fine defect segmentation.

1.12 Monitor the Rule After Deployment

A fixed threshold is easy to monitor because its expected behavior is stable. Track foreground fraction, connected-component count, mean foreground intensity, and mean background intensity. Slow drift can reveal lamp aging or lens contamination before segmentation fails completely. A sudden full-frame mask can indicate exposure or camera-setting failure.

Define explicit validity checks. For example, reject a result when the foreground covers less than 1% or more than 70% of the inspection region, when no component lies in the expected area range, or when the target centroid jumps farther than physically possible between synchronized frames. Thresholding creates the mask; operational monitoring decides whether that mask is credible enough for robot action.

2. How to Use Image Thresholding in the Telekinesis Agentic OS

Telekinesis provides global thresholding as the Cornea Skill segment_image_using_threshold. The code below loads the documented parts image, applies a binary threshold at intensity 45, and inspects the returned label map.

from telekinesis import cornea, datatypes

image = datatypes.Image.from_url(
    url="https://assets.telekinesis.ai/examples/v1/images/nuts_scattered.jpg"
).to_grayscale()

mask = cornea.segment_image_using_threshold(
    image=image,
    min_value=45,
    max_value=255,
    threshold_type="binary",
)

print(mask.shape)
print(mask.label_codes)
print(mask.number_of_labels)

The output shows the result of one fixed comparison. If foreground and background are reversed, use binary_inv; do not silently invert downstream assumptions without documenting which label represents the physical target.

Binary mask produced by manual global thresholding

The mask is meaningful only under the intensity convention and lighting conditions used to choose the threshold.

Try it out

Build with image thresholding in Cornea

See the current comparison modes, parameter definitions, output schema, and examples.

Open thresholding docs

Runnable example

Run the thresholding example

Open the exact Python script used to apply a fixed global threshold with Cornea.

View Python example

3. Benchmarking

Benchmark the complete operating envelope, not only nominal accuracy. Sweep the threshold across a validation set and plot precision, recall, IoU, component count, and centroid error. Repeat under minimum and maximum permitted illumination, exposure, material finish, and background variation. Also record latency, although basic global thresholding is usually negligible compared with image transfer and service overhead.

4. Where to Go Next?

5. References