Image & Point Cloud Fundamentals · Fundamentals

What is an Image?

An image is a rectangular grid of measurements captured from a scene. Learn how pixels, channels, resolution, and numeric ranges represent the visual input used by robot perception algorithms.

A robot camera does not deliver objects, edges, or meaning. It delivers numbers arranged in a grid. Understanding that grid makes every later vision topic—from thresholding to neural networks—much easier to reason about.

1. What is an Image?

An image is a rectangular grid of measurements, called pixels, recorded from a scene. Each pixel describes the light measured at one small image location.

An image represented as a grid of pixels

An image can be represented as a tensor

I∈RH×W×C.I\in\mathbb{R}^{H\times W\times C}.

II is the image, HH is its height in pixels, WW is its width in pixels, and CC is its number of channels. A grayscale image usually has one channel. An RGB image has three channels containing red, green, and blue values.

The position of a pixel is commonly written as (u,v)(u,v), where uu is the horizontal image coordinate and vv is the vertical coordinate. Unlike a point in the robot’s workspace, (u,v)(u,v) does not contain physical distance. It identifies a camera ray: somewhere along that ray, light from the scene reached the pixel.

1.1 Pixels Store Measurements, Not Objects

For an eight-bit RGB image, one pixel may be stored as

I(u,v)=(48,163,219).I(u,v)=(48,163,219).

The three numbers are the red, green, and blue channel values at pixel (u,v)(u,v). Each lies between 0 and 255. Machine-learning pipelines often normalize them to the range 0 to 1, but normalization does not add information; it only changes the numeric scale.

Those values do not say “blue cable.” That interpretation must come from an algorithm that uses surrounding pixels, learned visual patterns, or a known color rule.

1.2 Resolution and Field of View Are Different

Resolution tells you how many samples the image contains. A 1920×1080 image has more pixels than a 640×480 image. It does not automatically see a wider area. Field of view depends on the camera lens and sensor geometry; resolution controls how finely that view is sampled.

This distinction matters for robotics. A high-resolution camera pointed too far from a tiny component may still provide too few useful pixels. A wider lens may show more of the workcell while making every object smaller in the image.

1.3 What Changes the Pixel Values?

Pixel values depend on more than surface color. Illumination, exposure, gain, white balance, reflections, shadows, lens effects, and image compression all influence the recorded numbers. Two images of the same physical object can therefore differ significantly.

An image also lacks metric scale. A small nearby object can occupy the same number of pixels as a large distant object. To recover distance or physical size, the system needs additional geometry, such as a calibrated depth camera, stereo vision, or a known scene plane.

2. How Images Connect to Robot Perception

Most 2D vision algorithms begin with this grid and add structure:

OperationWhat it adds
ThresholdingA label for each pixel based on intensity or color
Contour detectionOrdered boundaries around connected regions
Object detectionCategories and rectangular image regions
SegmentationA semantic or instance label for each pixel
Depth estimationA distance associated with each image position

The central mental model is simple: an image says what light was measured and where in the image it was measured. It does not directly say what produced that light or where the source lies in 3D.

2.1 The Telekinesis Image Datatype

Telekinesis carries pixel arrays with datatypes.Image. It accepts grayscale (H, W), RGB (H, W, 3), and RGBA (H, W, 4) arrays. The object exposes properties such as height, width, channels, and dtype, so Skills can agree on the image contract.

import numpy as np
from telekinesis import datatypes

pixels = np.zeros((480, 640, 3), dtype=np.uint8)
image = datatypes.Image(pixels)

print(image.height, image.width, image.channels)  # 480 640 3

The datatype preserves pixels and validates their layout. It does not attach camera geometry; focal length, distortion, and camera pose belong to calibration data.

3. Where to Go Next