Image & Point Cloud Fundamentals · Fundamentals

RGB vs HSV vs LAB vs YCrCb

RGB, HSV, LAB, and YCrCb describe the same pixel with different coordinates, making different segmentation problems easier. Learn what each channel means, how the transforms work, and which color space fits a robotics task.

Imagine a robot sorting colored parts from a tray. Under the workcell lights, a blue part is easy to isolate. When sunlight reaches the tray, the same part becomes brighter on one side and darker on the other. Its physical color has not changed, but its pixel values have. Choosing the right color space can turn this difficult thresholding problem into a much simpler one.

1. RGB vs HSV vs LAB vs YCrCb

RGB, HSV, LAB, and YCrCb are four coordinate systems for describing color. They do not reveal new objects or recover information that the camera never captured. Instead, each reorganizes the same pixel values so that a particular property—raw sensor intensity, hue, perceptual difference, or chrominance—is easier to reason about.

This distinction matters because color segmentation is fundamentally a geometry problem. A threshold accepts one region of a three-dimensional color space and rejects the rest. If the target pixels form a compact cluster in that space, a simple lower and upper bound can work well. If brightness and color are tangled together, the same rule may break as soon as the lighting changes.

1.1 One Pixel, Four Ways to Describe It

The same pixel, four coordinate systems

Camera pixelOne measured color
RGBLight mixture
RRed intensity
GGreen intensity
BBlue intensity

Brightness and chromatic color remain coupled.

HSVColor picker
HHue angle
SColorfulness
VBrightness

Hue is separated, but becomes unstable near gray.

LABPerceptual coordinates
L*Lightness
a*Green ↔ red
b*Blue ↔ yellow

Numeric distance better approximates perceived difference.

YCrCbVideo color differences
YLuma
CrRed − luma
CbBlue − luma

Chrominance can be selected independently of luma.

Changing color space does not create new image evidence. It reorganizes the same camera measurement so a useful decision boundary may become simpler.

The diagram begins with one camera pixel and follows it into four representations. RGB stores the amounts of red, green, and blue in the encoded image. HSV rearranges those values into hue, saturation, and value. LAB describes lightness and two opponent color axes. YCrCb separates a luma signal from red- and blue-difference signals.

None is universally best. The useful question is: which coordinates keep the property I care about compact while pushing likely distractors away?

There is another practical consequence. Converting an RGB image does not make it immune to illumination. If a colored lamp changes the light reaching a surface, every color space changes because the original measurement changed. A better representation may isolate one kind of variation, but it cannot repeal image formation.

Interactive color coordinates

Describe the same pixel in four color spaces

#3b82f6
selected pixel
after brightness
RGBencoded channel intensities
HSVhue · saturation · value
LABlightness · green/red · blue/yellow
YCrCbluma · red difference · blue difference
Increase and decrease brightness. RGB channels and lightness move strongly, while hue or chroma may remain more stable—but no transform makes color independent of real illumination.

1.2 RGB: Work With the Camera’s Stored Channels

Most digital images arrive as three encoded channels:

RGB image split into three channel planesThree aligned image planes are stacked with a slight offset. They store blue, green, and red intensity, whose corresponding pixels combine into one color image. Values may use an eight-bit range from zero to 255 or a normalized range from zero to one.B · blue planeG · green planeR · red planecombineRGB color imagepixel = (R, G, B)8-bit image: 0–255normalized for ML: 0–1
An RGB image is three aligned intensity planes. The value at the same pixel location in the red, green, and blue planes is combined to produce the displayed color. Diagram is schematic; displayed shapes are not exact device gamuts.

Read the three offset squares as aligned images stacked behind one another, not as three different camera views. The marked location is the same pixel in every plane. Its red intensity comes from one layer, its green intensity from another, and its blue intensity from the third. Combining those three numbers produces the color pixel shown in the reconstructed image.

pRGB=[RGB]T.\mathbf{p}_{RGB} = \begin{bmatrix}R & G & B\end{bmatrix}^{\mathsf T}.

Here, RR, GG, and BB are the red, green, and blue channel values, and pRGB\mathbf{p}_{RGB} is the pixel represented as a three-dimensional vector. In an eight-bit image, each channel is usually stored between 0 and 255. Machine-learning pipelines often divide those values by 255 so each channel lies between 0 and 1. This normalization changes the numeric scale, not the underlying color.

RGB is intuitive when the task is already expressed in camera values. If a backlit inspection station always produces a bright red indicator against a dark enclosure, direct bounds such as R>180R>180, G<90G<90, and B<90B<90 may be all that is needed. There is no conversion to explain, and the accepted region is easy to inspect channel by channel.

The weakness appears when illumination changes. A darker red surface usually has lower values in all three channels than the same surface in bright light. Brightness and chromatic content are therefore coupled. A rectangular RGB threshold that includes both versions may become so wide that it also admits brown, orange, or gray pixels.

RGB is a good first choice when lighting, exposure, and white balance are controlled, or when one raw channel difference is itself the signal. It is a poor default when the desired rule sounds like “any reasonably bright shade of this hue.”

1.3 HSV: Turn Color Into Hue, Colorfulness, and Brightness

HSV is designed around the way people often operate a color picker. Hue describes a position around a color wheel, saturation describes how far the color is from gray, and value is the largest RGB component.

HSV continuous inverted cone with a cutaway sliceHue wraps continuously around a circular top, saturation runs from the white center to the colored rim, and value rises from a black apex. A removed wedge exposes the saturation and value directions.Hue HV = 1Saturation Swhite / gray axisconstant V sliceV = 0 · black
A smooth teaching visualization of HSV: hue wraps around the cone, saturation grows from the gray axis to the rim, and constant-value slices shrink toward black. The exact RGB-derived model in Smith’s paper has hexagonal rather than circular slices. Diagram is schematic; displayed shapes are not exact device gamuts.

Read the rounded cutaway cone in three motions. Move around the circular rim to change hue, move from the central gray axis toward the boundary to increase saturation, and move upward to increase value. The missing wedge exposes a constant-value slice inside the cone. These slices become smaller as value falls until the entire space converges at the black apex, where V=0V=0. This is why hue and saturation carry no visible distinction at black.

The smooth cone is a teaching visualization. In Smith’s original formulation, projecting the RGB cube produces a hexcone, so its constant-value sections are hexagons. Rounding those sections into circles makes hue and saturation easier to see, but it should not be mistaken for the exact boundary of the RGB-derived HSV gamut.

For normalized RR, GG, and BB, first define

Cmax=max⁡(R,G,B),Cmin=min⁡(R,G,B),Δ=Cmax−Cmin.C_{max}=\max(R,G,B),\qquad C_{min}=\min(R,G,B),\qquad \Delta=C_{max}-C_{min}.

CmaxC_{max} and CminC_{min} are the largest and smallest channel values, while Δ\Delta is their spread. Value and saturation are then

V=Cmax,S={0,Cmax=0,Δ/Cmax,Cmax>0.V=C_{max},\qquad S=\begin{cases} 0,&C_{max}=0,\\ \Delta/C_{max},&C_{max}>0. \end{cases}

VV is the HSV brightness coordinate and SS is the saturation. Hue HH is computed from which RGB channel is largest and from the relative channel differences. It is an angle around a circle: red lies near both the beginning and end of the stored hue range.

This organization makes many robotics rules natural. To find a blue cable, keep hue inside a blue interval, reject nearly gray pixels with a minimum saturation, and allow value to cover the expected shadow-to-highlight range. The threshold now says what the engineer means.

HSV has two traps. First, hue is circular. A red interval may need two ranges because red wraps around the numeric endpoint. Second, hue becomes unstable when saturation is close to zero. If RR, GG, and BB are nearly equal, tiny noise can cause a large hue jump even though the pixel still looks gray. Saturation should therefore be part of any hue-based decision.

Use HSV when a familiar named hue is the main signal and brightness should be handled separately. Test it carefully on pale objects, glare, deep shadow, and colors near the hue wrap-around.

1.4 LAB: Measure Lightness and Opponent Color

CIELAB was designed so that numeric distances correspond more closely to perceived color differences than raw RGB distances do. Its three coordinates are:

  • L∗L^*: perceptual lightness, from dark to light.
  • a∗a^*: an opponent axis from green toward red.
  • b∗b^*: an opponent axis from blue toward yellow.
CIELAB opponent axesLightness rises vertically while the horizontal plane spans green to red on a star and blue to yellow on b star.+a* red−a* green+b* yellow−b* blueL* = 100L* = 0ΔE neighborhood
LAB separates lightness from two opponent color axes. Equal geometric steps are intended to correspond more closely to equal perceived color changes. Diagram is schematic; displayed shapes are not exact device gamuts.

The vertical axis in the diagram changes lightness. The horizontal plane changes chromatic content: green opposes red along a∗a^*, while blue opposes yellow along b∗b^*. A tolerance around a reference color can therefore be pictured as a neighborhood in this three-dimensional space.

LAB is not computed directly from stored RGB with one simple subtraction. The RGB values must first be interpreted using their RGB color space, linearized, converted through CIE XYZ, and then transformed relative to a reference white. In the XYZ-to-LAB stage,

L∗=116f(Y/Yn)−16,L^*=116f(Y/Y_n)-16, a∗=500[f(X/Xn)−f(Y/Yn)],a^*=500\left[f(X/X_n)-f(Y/Y_n)\right], b∗=200[f(Y/Yn)−f(Z/Zn)].b^*=200\left[f(Y/Y_n)-f(Z/Z_n)\right].

XX, YY, and ZZ are CIE tristimulus values for the pixel; XnX_n, YnY_n, and ZnZ_n describe the reference white; and f(⋅)f(\cdot) is the standard CIELAB piecewise cube-root function. L∗L^*, a∗a^*, and b∗b^* are the resulting LAB coordinates.

The simplest LAB color difference is

ΔEab=(ΔL∗)2+(Δa∗)2+(Δb∗)2.\Delta E_{ab}=\sqrt{(\Delta L^*)^2+(\Delta a^*)^2+(\Delta b^*)^2}.

ΔL∗\Delta L^*, Δa∗\Delta a^*, and Δb∗\Delta b^* are the channel differences between two colors, and ΔEab\Delta E_{ab} is their Euclidean distance in LAB. More modern color-difference formulas improve perceptual uniformity, but this equation already explains LAB’s attraction: “close in color” becomes a meaningful geometric idea.

For robot inspection, LAB is useful when the question is whether a painted surface, food product, label, or manufactured part differs from a reference color by an acceptable amount. Thresholding a∗a^* and b∗b^* can also reduce sensitivity to lightness changes.

The trade-off is that LAB depends on the assumed RGB encoding and reference white. Camera auto white balance can move colors before conversion, and LAB is only approximately perceptually uniform. It is a better coordinate system for color difference—not a calibration substitute.

1.5 YCrCb: Separate Luma From Color Differences

YCrCb comes from digital video and image coding. It stores a luma channel YY plus two chroma differences: CrCr for red relative to luma and CbCb for blue relative to luma. For a BT.601-style transform,

YCrCb luma and chroma coordinatesLuma rises vertically through several chroma planes whose axes measure red difference and blue difference.Y luma+Cr+Cbbrightdarkfixed chroma range,wide luma range
YCrCb places luma on one axis and color differences on a chroma plane. Widening Y while constraining Cr and Cb accepts related chroma across several brightness levels. Diagram is schematic; displayed shapes are not exact device gamuts.

Each horizontal slice in the diagram holds chroma at one luma level. The neutral gray axis runs through the center, where both color differences are neutral. A threshold can remain narrow on the chroma plane while extending through several luma slices—the geometric reason YCrCb can tolerate some brightness variation.

Y′=0.299R′+0.587G′+0.114B′.Y' = 0.299R' + 0.587G' + 0.114B'.

R′R', G′G', and B′B' are gamma-encoded RGB components, and Y′Y' is encoded luma. In a common full-range eight-bit implementation, the remaining channels can be approximated by

Cr=0.713(R−Y)+128,Cb=0.564(B−Y)+128.Cr=0.713(R-Y)+128,\qquad Cb=0.564(B-Y)+128.

CrCr and CbCb are the red- and blue-difference components, while 128 is the neutral chroma offset used for unsigned storage. Exact coefficients, range scaling, and channel order vary between standards and libraries, so thresholds must be calibrated in the same implementation used during deployment.

YCrCb is especially useful when chrominance should be selected across a broad luma range. Classical skin-region proposals often constrain CrCr and CbCb while allowing YY to vary. The same idea can help with warm-colored materials or video-derived imagery.

This separation is not semantic understanding. Cardboard, leather, wood, and skin can overlap in chrominance. YCrCb should provide a color proposal, not proof of an object or person. Also note the naming trap: standards usually say YCbCr, while some computer-vision APIs expose the array in Y, Cr, Cb order.

1.6 What Actually Happens When the Lighting Changes?

A simple image-formation model helps explain why no color space always wins:

Ic(x)=g Ec(x) ρc(x)+nc(x).I_c(x)=g\,E_c(x)\,\rho_c(x)+n_c(x).

Ic(x)I_c(x) is the measured value at pixel xx in color channel cc; gg represents exposure and camera gain; Ec(x)E_c(x) is the illuminant reaching the surface; ρc(x)\rho_c(x) is the surface reflectance; and nc(x)n_c(x) is sensor and processing noise.

If only gg changes, brightness-like coordinates move strongly while ratios or chroma may remain more stable. HSV and YCrCb can help separate that variation. If EcE_c changes differently by channel—as it does under a colored lamp—the apparent hue and chroma also move. If a highlight appears, the measured color becomes a mixture of surface and illuminant. No coordinate transform restores the hidden reflectance exactly.

This is why good segmentation begins with the workcell: diffuse lighting, fixed exposure, stable white balance, and a deliberate background often improve every color space more than another round of threshold tuning.

1.7 How the Same Threshold Creates Different Decisions

The Telekinesis color segmentation Skills use lower and upper bounds. For a three-channel pixel q=(q1,q2,q3)\mathbf q=(q_1,q_2,q_3), the mask is

M(x,y)=1 ⁣[li≤qi(x,y)≤ui for i∈{1,2,3}].M(x,y)=\mathbb{1}\!\left[l_i\le q_i(x,y)\le u_i\ \text{for }i\in\{1,2,3\}\right].

M(x,y)M(x,y) is one for an accepted pixel and zero otherwise; lil_i and uiu_i are the lower and upper limits for channel ii; and 1[⋅]\mathbb{1}[\cdot] is the indicator function.

This rule always creates an axis-aligned box in the chosen space. But when that box is transformed back into RGB, its shape is different for HSV, LAB, and YCrCb. Changing color space therefore changes the family of color regions that a simple threshold can describe.

There is one important exception to the simple-box intuition: hue is circular. A single interval cannot cross the endpoint. Red commonly requires two HSV masks that are combined afterward.

1.8 Which Color Space Should You Choose?

Start withWhen the task sounds likeMain advantageMain caution
RGB“The raw red channel is high under fixed lighting”Direct, fast, easy to inspectBrightness and color are coupled
HSV“Find this hue across moderate brightness changes”Human-readable hue and saturationHue wraps and fails near gray
LAB“Keep colors perceptually close to this reference”Useful lightness/color separation and distanceDepends on color management and reference white
YCrCb“Select this chrominance across a broad luma range”Luma is separated from color differencesConventions vary; distractors may share chroma

The table is a starting point, not a verdict. Sample pixels from real deployment images, plot or summarize their distributions, choose a candidate space, and evaluate the resulting mask on scenes that include shadows, highlights, target absence, and confusing materials.

1.9 A Practical Selection Process

Begin with the physical requirement. If lighting can be fixed and the target is distinct, try RGB because its failure modes are easiest to see. If the requirement is expressed as a named color, try HSV and explicitly handle low saturation. If acceptable color difference is the engineering quantity, use LAB. If luma should vary while chrominance stays constrained—especially in video-oriented or coarse skin-color processing—try YCrCb.

Then test, rather than debate, the representations. Collect a small validation set from the actual camera and lens. Include every lighting mode, target material, background, exposure setting, and expected confounder. Tune thresholds on one subset and report intersection over union, false positives, and false negatives on another. A color space earns its place by making the deployed decision stable, not by being theoretically fashionable.

2. Comparing the Four Spaces in the Telekinesis Agentic OS

Telekinesis provides each representation as a Cornea segmentation Skill. The code below keeps the experiment consistent: one image is sent through all four Skills, while each set of bounds is calibrated separately in its own color space. The numeric bounds are illustrative and must be replaced with values sampled from your camera images.

from telekinesis import cornea, datatypes

image = datatypes.Image.from_url(
    url="https://assets.telekinesis.ai/examples/v1/images/wires_rgb.png"
)

masks = {
    "RGB": cornea.segment_image_using_rgb(
        image=image,
        lower_bound=(0, 0, 100),
        upper_bound=(140, 180, 255),
    ),
    "HSV": cornea.segment_image_using_hsv(
        image=image,
        lower_bound=(90, 70, 40),
        upper_bound=(140, 255, 255),
    ),
    "LAB": cornea.segment_image_using_lab(
        image=image,
        lower_bound=(20, 110, 60),
        upper_bound=(255, 170, 135),
    ),
    "YCrCb": cornea.segment_image_using_ycrcb(
        image=image,
        lower_bound=(0, 80, 135),
        upper_bound=(255, 150, 200),
    ),
}

for color_space, mask in masks.items():
    foreground_pixels = int((mask.labels == 1).sum())
    print(color_space, foreground_pixels)

Do not compare the four methods by copying one triplet of numbers between them. (100, 150, 200) means red, green, and blue in one call; hue, saturation, and value in another; and entirely different axes in LAB and YCrCb. Calibrate each space independently from the same labeled image set.

The loop prints the number of foreground pixels selected by each mask. Those counts are useful for catching an obviously empty or over-inclusive threshold, but they do not measure segmentation quality. Compare every returned mask with a labeled reference before deciding which color space works best.

Try it out

Build color segmentation pipelines with Cornea

Learn how Telekinesis represents segmentation images and how classical color masks fit into a robot vision pipeline.

Open the segmentation tutorial

3. How to Benchmark a Color Space Fairly

The fairest comparison holds everything except the color representation constant. Use the same source images and ground-truth masks, give every method a comparable threshold-tuning budget, and evaluate on images that were not used to select the bounds.

For predicted mask MM and ground-truth mask GG, intersection over union is

IoU(M,G)=∣M∩G∣∣M∪G∣.IoU(M,G)=\frac{|M\cap G|}{|M\cup G|}.

M∩GM\cap G is the set of pixels both masks call foreground, M∪GM\cup G is the set called foreground by either mask, and IoUIoU ranges from zero to one. Also report precision when false picks are expensive and recall when missing a target is dangerous.

Break results down by operating condition. A single average can hide that HSV succeeds in diffuse light but fails on desaturated highlights, or that YCrCb accepts wooden backgrounds. Record at least direct light, shadow, target absence, multiple camera exposures, and the most similar non-target materials.

Finally, measure the quantity that matters to the robot. A mask with slightly lower pixel IoU may still produce a more stable centroid or grasp region. Conversely, a visually pleasing mask may create a false connected component that sends the robot toward the wrong object. Pixel metrics and task success belong in the same evaluation.

4. Where to Go Next

5. References