Imagine a robot sorting colored parts from a tray. Under the workcell lights, a blue part is easy to isolate. When sunlight reaches the tray, the same part becomes brighter on one side and darker on the other. Its physical color has not changed, but its pixel values have. Choosing the right color space can turn this difficult thresholding problem into a much simpler one.
1. RGB vs HSV vs LAB vs YCrCb
RGB, HSV, LAB, and YCrCb are four coordinate systems for describing color. They do not reveal new objects or recover information that the camera never captured. Instead, each reorganizes the same pixel values so that a particular property—raw sensor intensity, hue, perceptual difference, or chrominance—is easier to reason about.
This distinction matters because color segmentation is fundamentally a geometry problem. A threshold accepts one region of a three-dimensional color space and rejects the rest. If the target pixels form a compact cluster in that space, a simple lower and upper bound can work well. If brightness and color are tangled together, the same rule may break as soon as the lighting changes.
1.1 One Pixel, Four Ways to Describe It
The same pixel, four coordinate systems
Brightness and chromatic color remain coupled.
Hue is separated, but becomes unstable near gray.
Numeric distance better approximates perceived difference.
Chrominance can be selected independently of luma.
The diagram begins with one camera pixel and follows it into four representations. RGB stores the amounts of red, green, and blue in the encoded image. HSV rearranges those values into hue, saturation, and value. LAB describes lightness and two opponent color axes. YCrCb separates a luma signal from red- and blue-difference signals.
None is universally best. The useful question is: which coordinates keep the property I care about compact while pushing likely distractors away?
There is another practical consequence. Converting an RGB image does not make it immune to illumination. If a colored lamp changes the light reaching a surface, every color space changes because the original measurement changed. A better representation may isolate one kind of variation, but it cannot repeal image formation.
Interactive color coordinates
Describe the same pixel in four color spaces
1.2 RGB: Work With the Camera’s Stored Channels
Most digital images arrive as three encoded channels:
Read the three offset squares as aligned images stacked behind one another, not as three different camera views. The marked location is the same pixel in every plane. Its red intensity comes from one layer, its green intensity from another, and its blue intensity from the third. Combining those three numbers produces the color pixel shown in the reconstructed image.
Here, , , and are the red, green, and blue channel values, and is the pixel represented as a three-dimensional vector. In an eight-bit image, each channel is usually stored between 0 and 255. Machine-learning pipelines often divide those values by 255 so each channel lies between 0 and 1. This normalization changes the numeric scale, not the underlying color.
RGB is intuitive when the task is already expressed in camera values. If a backlit inspection station always produces a bright red indicator against a dark enclosure, direct bounds such as , , and may be all that is needed. There is no conversion to explain, and the accepted region is easy to inspect channel by channel.
The weakness appears when illumination changes. A darker red surface usually has lower values in all three channels than the same surface in bright light. Brightness and chromatic content are therefore coupled. A rectangular RGB threshold that includes both versions may become so wide that it also admits brown, orange, or gray pixels.
RGB is a good first choice when lighting, exposure, and white balance are controlled, or when one raw channel difference is itself the signal. It is a poor default when the desired rule sounds like “any reasonably bright shade of this hue.”
1.3 HSV: Turn Color Into Hue, Colorfulness, and Brightness
HSV is designed around the way people often operate a color picker. Hue describes a position around a color wheel, saturation describes how far the color is from gray, and value is the largest RGB component.
Read the rounded cutaway cone in three motions. Move around the circular rim to change hue, move from the central gray axis toward the boundary to increase saturation, and move upward to increase value. The missing wedge exposes a constant-value slice inside the cone. These slices become smaller as value falls until the entire space converges at the black apex, where . This is why hue and saturation carry no visible distinction at black.
The smooth cone is a teaching visualization. In Smith’s original formulation, projecting the RGB cube produces a hexcone, so its constant-value sections are hexagons. Rounding those sections into circles makes hue and saturation easier to see, but it should not be mistaken for the exact boundary of the RGB-derived HSV gamut.
For normalized , , and , first define
and are the largest and smallest channel values, while is their spread. Value and saturation are then
is the HSV brightness coordinate and is the saturation. Hue is computed from which RGB channel is largest and from the relative channel differences. It is an angle around a circle: red lies near both the beginning and end of the stored hue range.
This organization makes many robotics rules natural. To find a blue cable, keep hue inside a blue interval, reject nearly gray pixels with a minimum saturation, and allow value to cover the expected shadow-to-highlight range. The threshold now says what the engineer means.
HSV has two traps. First, hue is circular. A red interval may need two ranges because red wraps around the numeric endpoint. Second, hue becomes unstable when saturation is close to zero. If , , and are nearly equal, tiny noise can cause a large hue jump even though the pixel still looks gray. Saturation should therefore be part of any hue-based decision.
Use HSV when a familiar named hue is the main signal and brightness should be handled separately. Test it carefully on pale objects, glare, deep shadow, and colors near the hue wrap-around.
1.4 LAB: Measure Lightness and Opponent Color
CIELAB was designed so that numeric distances correspond more closely to perceived color differences than raw RGB distances do. Its three coordinates are:
- : perceptual lightness, from dark to light.
- : an opponent axis from green toward red.
- : an opponent axis from blue toward yellow.
The vertical axis in the diagram changes lightness. The horizontal plane changes chromatic content: green opposes red along , while blue opposes yellow along . A tolerance around a reference color can therefore be pictured as a neighborhood in this three-dimensional space.
LAB is not computed directly from stored RGB with one simple subtraction. The RGB values must first be interpreted using their RGB color space, linearized, converted through CIE XYZ, and then transformed relative to a reference white. In the XYZ-to-LAB stage,
, , and are CIE tristimulus values for the pixel; , , and describe the reference white; and is the standard CIELAB piecewise cube-root function. , , and are the resulting LAB coordinates.
The simplest LAB color difference is
, , and are the channel differences between two colors, and is their Euclidean distance in LAB. More modern color-difference formulas improve perceptual uniformity, but this equation already explains LAB’s attraction: “close in color” becomes a meaningful geometric idea.
For robot inspection, LAB is useful when the question is whether a painted surface, food product, label, or manufactured part differs from a reference color by an acceptable amount. Thresholding and can also reduce sensitivity to lightness changes.
The trade-off is that LAB depends on the assumed RGB encoding and reference white. Camera auto white balance can move colors before conversion, and LAB is only approximately perceptually uniform. It is a better coordinate system for color difference—not a calibration substitute.
1.5 YCrCb: Separate Luma From Color Differences
YCrCb comes from digital video and image coding. It stores a luma channel plus two chroma differences: for red relative to luma and for blue relative to luma. For a BT.601-style transform,
Each horizontal slice in the diagram holds chroma at one luma level. The neutral gray axis runs through the center, where both color differences are neutral. A threshold can remain narrow on the chroma plane while extending through several luma slices—the geometric reason YCrCb can tolerate some brightness variation.
, , and are gamma-encoded RGB components, and is encoded luma. In a common full-range eight-bit implementation, the remaining channels can be approximated by
and are the red- and blue-difference components, while 128 is the neutral chroma offset used for unsigned storage. Exact coefficients, range scaling, and channel order vary between standards and libraries, so thresholds must be calibrated in the same implementation used during deployment.
YCrCb is especially useful when chrominance should be selected across a broad luma range. Classical skin-region proposals often constrain and while allowing to vary. The same idea can help with warm-colored materials or video-derived imagery.
This separation is not semantic understanding. Cardboard, leather, wood, and skin can overlap in chrominance. YCrCb should provide a color proposal, not proof of an object or person. Also note the naming trap: standards usually say YCbCr, while some computer-vision APIs expose the array in Y, Cr, Cb order.
1.6 What Actually Happens When the Lighting Changes?
A simple image-formation model helps explain why no color space always wins:
is the measured value at pixel in color channel ; represents exposure and camera gain; is the illuminant reaching the surface; is the surface reflectance; and is sensor and processing noise.
If only changes, brightness-like coordinates move strongly while ratios or chroma may remain more stable. HSV and YCrCb can help separate that variation. If changes differently by channel—as it does under a colored lamp—the apparent hue and chroma also move. If a highlight appears, the measured color becomes a mixture of surface and illuminant. No coordinate transform restores the hidden reflectance exactly.
This is why good segmentation begins with the workcell: diffuse lighting, fixed exposure, stable white balance, and a deliberate background often improve every color space more than another round of threshold tuning.
1.7 How the Same Threshold Creates Different Decisions
The Telekinesis color segmentation Skills use lower and upper bounds. For a three-channel pixel , the mask is
is one for an accepted pixel and zero otherwise; and are the lower and upper limits for channel ; and is the indicator function.
This rule always creates an axis-aligned box in the chosen space. But when that box is transformed back into RGB, its shape is different for HSV, LAB, and YCrCb. Changing color space therefore changes the family of color regions that a simple threshold can describe.
There is one important exception to the simple-box intuition: hue is circular. A single interval cannot cross the endpoint. Red commonly requires two HSV masks that are combined afterward.
1.8 Which Color Space Should You Choose?
| Start with | When the task sounds like | Main advantage | Main caution |
|---|---|---|---|
| RGB | “The raw red channel is high under fixed lighting” | Direct, fast, easy to inspect | Brightness and color are coupled |
| HSV | “Find this hue across moderate brightness changes” | Human-readable hue and saturation | Hue wraps and fails near gray |
| LAB | “Keep colors perceptually close to this reference” | Useful lightness/color separation and distance | Depends on color management and reference white |
| YCrCb | “Select this chrominance across a broad luma range” | Luma is separated from color differences | Conventions vary; distractors may share chroma |
The table is a starting point, not a verdict. Sample pixels from real deployment images, plot or summarize their distributions, choose a candidate space, and evaluate the resulting mask on scenes that include shadows, highlights, target absence, and confusing materials.
1.9 A Practical Selection Process
Begin with the physical requirement. If lighting can be fixed and the target is distinct, try RGB because its failure modes are easiest to see. If the requirement is expressed as a named color, try HSV and explicitly handle low saturation. If acceptable color difference is the engineering quantity, use LAB. If luma should vary while chrominance stays constrained—especially in video-oriented or coarse skin-color processing—try YCrCb.
Then test, rather than debate, the representations. Collect a small validation set from the actual camera and lens. Include every lighting mode, target material, background, exposure setting, and expected confounder. Tune thresholds on one subset and report intersection over union, false positives, and false negatives on another. A color space earns its place by making the deployed decision stable, not by being theoretically fashionable.
2. Comparing the Four Spaces in the Telekinesis Agentic OS
Telekinesis provides each representation as a Cornea segmentation Skill. The code below keeps the experiment consistent: one image is sent through all four Skills, while each set of bounds is calibrated separately in its own color space. The numeric bounds are illustrative and must be replaced with values sampled from your camera images.
from telekinesis import cornea, datatypes
image = datatypes.Image.from_url(
url="https://assets.telekinesis.ai/examples/v1/images/wires_rgb.png"
)
masks = {
"RGB": cornea.segment_image_using_rgb(
image=image,
lower_bound=(0, 0, 100),
upper_bound=(140, 180, 255),
),
"HSV": cornea.segment_image_using_hsv(
image=image,
lower_bound=(90, 70, 40),
upper_bound=(140, 255, 255),
),
"LAB": cornea.segment_image_using_lab(
image=image,
lower_bound=(20, 110, 60),
upper_bound=(255, 170, 135),
),
"YCrCb": cornea.segment_image_using_ycrcb(
image=image,
lower_bound=(0, 80, 135),
upper_bound=(255, 150, 200),
),
}
for color_space, mask in masks.items():
foreground_pixels = int((mask.labels == 1).sum())
print(color_space, foreground_pixels)
Do not compare the four methods by copying one triplet of numbers between them. (100, 150, 200) means red, green, and blue in one call; hue, saturation, and value in another; and entirely different axes in LAB and YCrCb. Calibrate each space independently from the same labeled image set.
The loop prints the number of foreground pixels selected by each mask. Those counts are useful for catching an obviously empty or over-inclusive threshold, but they do not measure segmentation quality. Compare every returned mask with a labeled reference before deciding which color space works best.
3. How to Benchmark a Color Space Fairly
The fairest comparison holds everything except the color representation constant. Use the same source images and ground-truth masks, give every method a comparable threshold-tuning budget, and evaluate on images that were not used to select the bounds.
For predicted mask and ground-truth mask , intersection over union is
is the set of pixels both masks call foreground, is the set called foreground by either mask, and ranges from zero to one. Also report precision when false picks are expensive and recall when missing a target is dangerous.
Break results down by operating condition. A single average can hide that HSV succeeds in diffuse light but fails on desaturated highlights, or that YCrCb accepts wooden backgrounds. Record at least direct light, shadow, target absence, multiple camera exposures, and the most similar non-target materials.
Finally, measure the quantity that matters to the robot. A mask with slightly lower pixel IoU may still produce a more stable centroid or grasp region. Conversely, a visually pleasing mask may create a false connected component that sends the robot toward the wrong object. Pixel metrics and task success belong in the same evaluation.
4. Where to Go Next
5. References
- Smith, A. R. “Color Gamut Transform Pairs”, SIGGRAPH Computer Graphics, 1978.
- International Commission on Illumination. “CIELAB colour space”, International Lighting Vocabulary.
- International Telecommunication Union. “Recommendation ITU-R BT.601: Studio Encoding Parameters of Digital Television”, 2011.
- International Color Consortium. “sRGB Characterization Data”.
- Telekinesis. “Image Segmentation”, Cornea tutorial.



