Image Segmentation · Cornea

What is Flood Fill Segmentation?

Flood fill segmentation solves the problem of extracting one connected, visually similar region from a known interior point. Learn seed-based region growing, tolerance and connectivity, failure modes, and how to use it in the Telekinesis Agentic OS.

Imagine a robot looking at a tray whose background is mostly one color. You know the top-left pixel belongs to that background, but a global color threshold also finds disconnected objects elsewhere with similar color. Flood fill expresses the missing requirement: select pixels that are both similar and connected to the known background point.

1. What is Flood Fill Segmentation?

Flood fill is a seeded region-growing algorithm that extracts pixels reachable from a chosen seed through a path of accepted neighbors. It solves segmentation tasks in which one representative interior point is known and connectivity is part of the definition.

The familiar paint-bucket tool uses the same idea. A fill starts where the user clicks, spreads through similar neighboring pixels, and stops at boundaries that fail the similarity test. In robotics, the filled region becomes a mask rather than merely receiving a new display color.

1.1 The Complete Flood Fill Pipeline

Seed + similarity + connectivity → one region

ImageGrayscale or color pixels
Seed + toleranceKnown point inside the desired region
Initialize frontierStart from the seed pixel
Test neighborsCheck connection and similarity to seed value
Grow until stableAdd accepted neighbors and repeat
Connected region maskReachable similar pixels vs. remainder

A distant pixel can match the seed perfectly and still remain excluded when no accepted path connects it to the seed.

Flood fill is seeded region growing: it repeatedly visits neighboring pixels, accepts those within tolerance of the seed value, and stops when no new acceptable neighbor is reachable. Adapted as an explanatory diagram fromTelekinesis Flood Fill documentation.

Flood fill starts from one pixel that the engineer already believes belongs to the desired region. This seed might be placed by a user, selected from the center of a detection, or chosen from known workcell geometry. A tolerance defines how different a neighboring pixel may be while still joining the region.

The seed enters a list of locations waiting to be explored. The algorithm removes one location, checks its neighbors, and adds every acceptable neighbor that has not already been visited. Those new pixels are then explored in the same way, so the region spreads outward through connected, similar pixels.

Growth stops naturally when no acceptable neighbor remains. A strong color or intensity boundary acts like a wall because pixels across it fail the tolerance test. The output is therefore the connected region reachable from the seed, not every pixel in the image with a matching value.

That last point separates flood fill from thresholding. Thresholding can select two disconnected regions because they share an intensity. Flood fill selects only the region connected by an unbroken path from the seed. The next section turns this intuitive spreading process into a graph model.

1.2 Model the Image as a Graph

Think of each pixel as a graph node. Edges connect neighboring pixels. Under four-connectivity, a pixel connects horizontally and vertically. Under eight-connectivity, diagonal neighbors also count.

Let s=(xs,ys)s=(x_s,y_s) be the seed. Let N(p)\mathcal N(p) be the neighborhood of pixel pp. The grown region RR begins as

R0={s}.R_0=\{s\}.

At each step, acceptable neighbors are added:

Rk+1=Rk∪{q∈N(p):p∈Rk,  D(I(q),I(s))≤τ}.R_{k+1}=R_k\cup \left\{q\in\mathcal N(p):p\in R_k,\;D(I(q),I(s))\le\tau\right\}.

I(q)I(q) is the value or color at candidate pixel qq, I(s)I(s) is the seed value, DD is a difference measure, and τ\tau is the tolerance. Growth stops when Rk+1=RkR_{k+1}=R_k.

The Telekinesis documentation defines tolerance relative to the seed pixel value. That fixed reference prevents the region from gradually drifting through many small neighbor-to-neighbor changes.

1.3 Why Connectivity Changes the Answer

Suppose two blue regions have identical color but are separated by a red stripe. A blue color threshold selects both. Flood fill seeded in the first blue region selects only the first because the red stripe breaks every acceptable path.

Connectivity also determines whether diagonal contact joins regions. Two pixels touching only at a corner are disconnected under four-connectivity but connected under eight-connectivity. The Cornea Skill does not expose a connectivity parameter, so treat its deployed neighborhood rule as part of the implementation contract and validate corner-touching cases if they matter.

1.4 What Tolerance Means

With tolerance zero, only pixels exactly matching the seed are accepted. Real images contain sensor noise, compression, texture, and illumination variation, so a nonzero tolerance is usually necessary.

For scalar intensity, a simple similarity rule is

D(I(q),I(s))=∣I(q)−I(s)∣.D(I(q),I(s))=|I(q)-I(s)|.

For color, implementations may compare channels or use another vector difference. The public Skill exposes one integer tolerance; use the documentation and empirical tests to determine how that tolerance behaves for your image type.

A tolerance that is too small fragments the intended region. One that is too large leaks across weak boundaries. Tuning is therefore about the narrowest physical bottleneck between desired and undesired pixels, not only about average region color.

Interactive region growing

Choose a seed, tolerance, and connectivity

0 pixels selected
Connectivity
Seed value —

Click any square to move the seed. A pixel is selected only when it is both similar enough to the seed and connected through other accepted pixels.

selected rejected seed
Notice that a distant pixel can have the right value and remain excluded when no valid path reaches it.

1.5 Seed Choice Is Part of the Algorithm

The seed is not a harmless starting detail. It defines the reference value and the connected component to be returned. A seed placed on glare, a shadow, an edge blend, or the wrong object can produce a completely different mask.

Good seeds come from known geometry or upstream perception: a tray corner known to be background, the center of a detector box, a manually taught pixel, or a projected robot-frame point. If the seed is predicted, validate that it lies inside the intended region before trusting the fill.

For a moving scene, timestamp the image and seed source together. A seed derived from an older frame may land on a different object after motion.

1.6 Queue-Based Growth Without Recursion

The algorithm can be implemented with breadth-first search:

  1. Add the seed to a queue and mark it visited.
  2. Remove one pixel from the queue.
  3. Test each permitted neighbor.
  4. Add every unvisited, seed-similar neighbor.
  5. Stop when the queue is empty.

Depth-first search produces the same connected region under the same acceptance rule, but naive recursive implementations risk overflowing the call stack on large images. Queue- or explicit-stack-based implementations keep memory behavior under program control. Scanline variants reduce overhead by processing runs of adjacent pixels.

1.7 Flood Fill Has No Training Loss

Flood fill contains no learned weights, dataset, or training phase. The seed, tolerance, and neighborhood rule are configuration inputs. There is therefore no loss function.

Evaluate the resulting mask against annotations using IoU, precision, and recall. Add seed-perturbation tests: move the seed a few pixels in every direction and measure how often the chosen physical region changes. For boundary-sensitive tasks, report leakage across the weakest real boundary.

1.8 Common Failure Modes

Leakage through a bridge. One chain of accepted pixels can connect the desired region to a large distractor. A narrow reflection or anti-aliased edge may provide that bridge.

Fragmentation. Texture or shading pushes valid pixels beyond tolerance, breaking connectivity and stopping growth early.

Wrong seed. The algorithm correctly grows the wrong region.

Color ambiguity inside the region. A multicolored object cannot be captured by one seed-relative tolerance even if its shape is connected.

Boundary contact. Background seeded at an image corner is convenient only when that corner is guaranteed to show background. A foreground object touching the corner invalidates the assumption.

These are interpretable failures. Visualize the seed, difference-to-seed image, frontier boundary, and final mask during commissioning.

1.9 Flood Fill Versus Other Segmentation Methods

MethodUses intensity/colorUses connectivityRequires seedMain question
Global thresholdYesNoNo“Which pixels cross this cutoff?”
HSV/RGB rangeYesNoNo“Which pixels have this color?”
Flood fillYesYesYes“Which similar region contains this point?”
ContoursUsually binary inputEncodes boundariesNo“What ordered outlines exist?”
SAM 3Learned semanticsProduces instance masksPrompt“Which pixels belong to this concept?”

Flood fill is ideal when location and continuity disambiguate appearance. It is not a replacement for semantic segmentation when the desired region contains varied appearance or the seed is unknown.

1.10 From Region Mask to Robot Action

Once background is filled, invert the mask to propose foreground objects. Once an object is filled, calculate its contour, centroid, principal axis, or area. Verify the result against expected size and shape before motion.

The mask remains image-space evidence. A centroid is not a metric pick point until camera calibration and depth or a known plane map it into the robot frame. A filled region is not proof of safe reachability, grasp quality, or collision clearance.

1.11 Fixed-Range and Floating-Range Growth

Region-growing implementations commonly use one of two similarity ideas. In fixed-range growth, every candidate is compared with the original seed value, as described by the Telekinesis Skill. In floating-range growth, a candidate may be compared with a neighboring accepted pixel or an evolving region statistic.

Floating range can follow a gradual intensity ramp: each step differs only slightly from the previous one. That flexibility also permits “region drift,” where a chain of small changes eventually reaches pixels very different from the seed. Fixed range prevents this drift but may stop early when legitimate shading moves far from the starting value.

Knowing which rule is deployed changes how tolerance should be interpreted. A tolerance of 10 under fixed range defines a stable band around the seed. The same local step tolerance under floating range can cross a much larger total intensity span.

1.12 Complexity and Memory

With a visited mask and explicit queue, each accepted pixel is processed once and each permitted neighbor is checked a bounded number of times. For an image containing NN pixels, worst-case time and memory are therefore O(N)O(N) when the fill reaches most of the frame.

This matters for high-resolution robot cameras. A background fill may legitimately cover millions of pixels. Recursive code can exhaust the call stack, while a queue consumes memory proportional to the active frontier. Scanline fill variants process horizontal runs to reduce queue overhead. The hosted Skill abstracts implementation details, but end-to-end latency and maximum image size still belong in deployment tests.

1.13 Multiple Seeds and Multiple Regions

One flood-fill call returns the connected region associated with one seed. If the task contains several disconnected regions of the same material, one seed cannot reach all of them. Engineers can supply multiple validated seeds in separate calls, or threshold first and use connected-component labeling to enumerate every region.

Multiple seeds also create conflict questions: what happens if their tolerances overlap, and which label owns the boundary? A marker-based watershed is often a better formalism when several seeds should compete for pixels. Flood fill remains clearest when the task asks for one known connected region.

1.14 Choose and Validate Tolerance Systematically

Sample intensities inside the desired region and along its closest confusing boundary. Measure the largest within-region difference from candidate seeds and the smallest difference across the boundary. A useful tolerance must include expected internal variation while remaining below the boundary contrast.

If those ranges overlap, tolerance alone cannot guarantee containment. Improve lighting, choose a more representative color space, add an edge barrier, or use a richer segmentation method. Increasing tolerance until one training image looks filled merely hides the overlap.

Sweep tolerance on held-out images and plot IoU plus leakage beyond the expected boundary. Repeat the sweep for plausible seed displacement. The selected pair—seed policy and tolerance—forms one configuration and should be validated together.

1.15 Production Guardrails

Before using a fill for motion, verify that the seed lies within the image and within an expected region of interest. Reject masks outside permitted area, bounding-box, or shape ranges. Track whether the mask touches forbidden image borders. If the fill represents background, unexpected holes or a sudden area reduction can signal an obstruction or illumination change.

Log the seed coordinate, seed pixel value, tolerance, foreground fraction, and timestamp. These fields make a surprising mask explainable after the fact. A deterministic classical algorithm is only as auditable as the inputs retained with its output.

2. How to Use Flood Fill in the Telekinesis Agentic OS

Telekinesis provides seeded region growing as the Cornea Skill segment_image_using_flood_fill. This example starts at the top-left pixel (0, 0) and includes connected neighbors within tolerance 10 of that seed value.

from telekinesis import cornea, datatypes

image = datatypes.Image.from_url(
    url="https://assets.telekinesis.ai/examples/v1/images/erode.jpg"
)

mask = cornea.segment_image_using_flood_fill(
    image=image,
    seed_point=(0, 0),  # [x, y]
    tolerance=10,
)

print(mask.shape)
print(mask.label_codes)
print(mask.number_of_labels)

The result contains the region reachable from the seed and the remaining image as separate labels. Log or draw the seed point alongside the mask so a reviewer can verify the premise of the operation.

Connected region selected by flood fill from a seed pixel

A matching color is insufficient: every selected pixel must also be reachable through accepted neighbors from the seed.

Try it out

Build with flood fill in Cornea

See the current seed and tolerance contract, output schema, and complete example.

Open flood fill docs

Runnable example

Run the flood fill example

Open the exact Python script used to grow a connected image region from a seed with Cornea.

View Python example

3. Benchmarking

Flood fill is a classical primitive rather than a learned model with a standard accuracy table. Benchmark tolerance sweeps on annotated regions, including several valid and invalid seed locations. Report IoU, leakage rate, fragmentation, sensitivity to seed displacement, and latency by image size. Test weak boundaries, diagonal contact, image-edge contact, shadows, texture, and target absence.

4. Where to Go Next?

5. References