Imagine a mobile robot moving through a warehouse. It must notice people, boxes, and vehicles quickly enough to react. YOLOX is an object detector built for this kind of fast visual recognition, and its design is a useful way to learn how modern real-time detection works.
1. What is YOLOX?
Suppose a mobile robot enters a warehouse aisle. One camera frame may contain a person nearby, several cartons at different distances, and a forklift partly hidden at the far end. The robot needs both identity and location: what is present and where it is in the image.
YOLOX is a real-time object detector that returns boxes, class labels, and confidence scores. It belongs to the YOLO family, whose central goal is to perform detection in one fast network pipeline.
YOLOX is described as a one-stage, anchor-free detector. One-stage means it predicts candidates directly from image features instead of using one learned stage to propose regions and another to classify them. Anchor-free means it predicts box geometry without starting from a catalogue of predefined rectangle shapes. Neither term changes the vocabulary: YOLOX still recognizes the classes it learned during training.
The paper develops this pipeline through three connected ideas: anchor-free prediction simplifies where boxes come from, a decoupled head separates the jobs of classification and localization, and SimOTA decides which predictions should learn from each annotated object.
1.1 The Complete Path From an Image to Final Detections
Image → filtered detections
SimOTA assigns positive locations during training; NMS resolves duplicates during inference.
YOLOX begins by passing the camera image through a backbone. The backbone replaces pixels with feature maps that respond to useful visual patterns. An SPP block helps those features include wider scene context, while the PAN feature pyramid combines several resolutions so small and large objects can both be represented.
The detection head examines many positions on those feature maps. It is anchor-free, so it does not begin with a catalogue of preset box shapes. At each candidate position, separate branches predict whether an object exists, which trained class it belongs to, and where its box lies.
Dense prediction means neighboring positions may all describe the same physical object. Score filtering first removes weak candidates. Non-maximum suppression then keeps the strongest box and removes highly overlapping duplicates, leaving the final detections used by the robot.
SimOTA appears in the training story rather than this inference path. It decides which candidate locations should learn from each labeled object. YOLOX is therefore a set of connected design choices: multi-scale features, anchor-free prediction, separated output branches, dynamic training assignment, and explicit duplicate removal. The next sections unpack them in the same order.
1.2 The Backbone and Feature Pyramid Preserve Several Scales
Follow one camera frame from the first diagram box. The image enters the backbone and spatial-pyramid-pooling path, which turns pixels into a hierarchy of feature maps. The paper begins its experiments from YOLOv3-SPP with DarkNet53, then uses a CSPNet backbone and PAN head for the scaled YOLOX variants. Stating both matters: “YOLOX” names the method and model family, not one immutable backbone configuration.
The PAN feature pyramid combines several resolutions. This matters because cartons near the robot and a forklift at the far end occupy very different numbers of pixels. Feature-pyramid representations let small objects use detailed high-resolution features while larger objects use features with wider context.
The Decoupled head then examines many positions across those multi-scale feature maps. At each candidate position it predicts box geometry, objectness, and class evidence. “Dense” simply means that the network checks a regular grid of possible locations rather than beginning with a short list of proposed objects.
1.3 Anchor-Free Prediction Defines Where Boxes Come From
To see why anchor-free prediction matters, consider an older anchor-based detector. At every grid location, it places several template rectangles with predefined widths and aspect ratios. The network learns how to shift and resize them. If the dataset changes from people and cars to long tools and tiny fasteners, those templates may need to be reconsidered.
YOLOX removes the template catalogue. It treats a feature-map location as a possible object centre and directly predicts the distances needed to describe the box. Relative to the paper’s anchor-based baseline, this reduces the predictions at each location from three to one.
The design is simpler, but it is not assumption-free. Input resolution determines how much detail exists, feature strides determine the spatial grid, and the training assignment determines which locations learn each object. Anchor-free removes one family of hand-designed choices; it does not remove the need for sound data and geometry.
1.4 The Decoupled Head Separates Three Different Questions
Once a candidate location has been chosen, the detector must answer different questions. “What is it?” depends on appearance and semantic cues. “Where exactly does it end?” depends on boundaries and geometry. Those tasks can prefer different features.
YOLOX therefore uses a decoupled head. The branches share the feature pyramid but separate near the output: one branch predicts class evidence, another predicts box geometry, and a third predicts objectness—whether any supported object is present at that location. This reduces direct interference between classification and localization.
A simplified final score can be understood as
Here, is the combined score for one class, is the location’s objectness estimate, and is its class score conditioned on an object being present.
This product explains why a location must look both object-like and class-consistent to receive a strong final score. It is a mental model, not a promise that the score is a calibrated probability. Thresholds still need to be validated on the robot’s own camera data.
1.5 SimOTA Decides Which Locations Should Learn Each Object
Anchor-free prediction and the decoupled head define what the network can predict, but training still needs to decide which grid locations are responsible for each annotated object. Dense prediction creates an ambiguity: many nearby locations may produce boxes around the same person. If all are treated as correct, duplicates dominate. If a poor location is chosen, the model learns from a weak example.
SimOTA resolves this in stages. A centre prior first removes implausible distant candidates. For the remaining candidates, the algorithm combines classification and regression losses into a matching cost. It then chooses a dynamic number of low-cost positives for each object. An object with several good candidates can supervise more locations than one with only a few reliable candidates.
This assignment exists only during training. It teaches the dense head which locations should become responsible for which objects. During inference, the trained network directly emits its candidates.
1.6 The Loss Function Trains Classification, Objectness, and Boxes
After SimOTA has assigned positive locations, the loss tells each branch how to improve. The YOLOX paper uses binary cross-entropy for the classification and objectness branches and IoU loss for box regression. A simplified total objective is
Here, is the total training loss; penalizes incorrect class scores; penalizes incorrect objectness scores; penalizes disagreement between predicted and target boxes; and the three terms weight the components.
Binary cross-entropy asks whether each class or objectness target should be active. IoU loss asks whether the predicted rectangle covers the same region as the annotation. The decoupled head gives these objectives separate branches, while joint optimization encourages the branches to agree on one useful detection.
This loss is distinct from SimOTA’s matching cost. SimOTA chooses the positive training locations. The loss updates the predictions made at those locations. Both disappear at inference time.
1.7 COCO Training Gives YOLOX Its Class Vocabulary
The paper trains the main YOLOX models from scratch on COCO train2017 for 300 epochs, with five warm-up epochs, stochastic gradient descent, a cosine learning-rate schedule, and multi-scale inputs sampled from 448 to 832 pixels. After strong augmentation was introduced, the authors found ImageNet initialization no longer beneficial for these experiments.
Training images use horizontal flips, colour jitter, multi-scale resizing, Mosaic, and MixUp. Mosaic combines transformed images into one training canvas; MixUp blends examples and their targets. Both are disabled for the last 15 epochs so the model finishes on samples closer to the real COCO distribution. Small YOLOX variants use weaker augmentation because aggressive mixing can remove too much useful signal at limited capacity.
COCO supplies 80 common object categories. A COCO-trained YOLOX can therefore detect people, vehicles, and everyday objects, but it does not automatically know a factory’s proprietary fastener or defect class. Supporting those targets requires labeled domain data and retraining or fine-tuning.
These objectives also explain common errors. High objectness with confused class scores means the model sees an object-like region but cannot identify it. A correct class with poor overlap means recognition succeeded while localization failed. Low values in every branch suggest that the feature pyramid did not preserve enough evidence or the scene differs too much from training.
1.8 Score Filtering and NMS Turn Candidates Into Final Detections
Training is now complete. At inference time, the network emits a large collection of candidates. Score filtering first removes candidates whose class and objectness evidence is too weak. Even among the remaining candidates, neighbouring grid locations can predict the same physical object.
Non-maximum suppression, or NMS, solves that duplicate problem. It sorts boxes by score, keeps the strongest, and removes weaker boxes that overlap it beyond a threshold. Overlap is measured with intersection over union:
Here, and are candidate box regions; is their intersection, is their union, and vertical bars denote region area.
A lower NMS threshold suppresses duplicates more aggressively but can erase one of two nearby parts. A higher threshold preserves crowded objects but may leave several boxes around one part. NMS finishes detection within one image; it is not tracking and cannot establish identity across frames.
Interactive post-processing
See score filtering and NMS remove candidate boxes
1.9 Feature Stride Sets a Practical Small-Object Limit
Each feature-pyramid level has a stride: one feature cell corresponds to several input pixels. A stride-8 map retains four times as many spatial positions as a stride-16 map across the same image area. Small targets depend heavily on these fine maps because they may occupy only a few cells at coarser resolution.
This creates a concrete camera-design question. If a fastener is 6 pixels wide after resizing, no threshold can recover detail that the input pipeline removed. Increasing input resolution may help, but it raises compute and latency. Moving the camera closer, narrowing the field of view, or cropping a known workspace may provide more useful pixels at lower cost.
Before comparing model variants, calculate the apparent pixel size of the smallest required target across the robot’s working distance. Detection architecture, optics, and camera placement form one system; a faster network cannot compensate for an object that was never adequately sampled.
1.10 Where YOLOX Can Fail
The whole story now connects. YOLOX extracts multi-scale features, predicts dense anchor-free candidates through separate output branches, learns their responsibility through SimOTA, and removes duplicates with NMS. This makes it fast and practical, but not infallible.
Small or blurred objects may not leave enough visual evidence. Unseen classes remain outside the learned vocabulary. Crowded objects can interfere during suppression, and a correct 2D box may still be too coarse for grasping. A robot should treat YOLOX as the localization stage of a larger perception system, not as a complete understanding of the scene.
2. How to Use YOLOX in the Telekinesis Agentic OS
Telekinesis provides YOLOX as the Retina Skill detect_objects_using_yolox. The code below loads an image, applies score and non-maximum-suppression thresholds, resolves the COCO category names, and prints each detection.
from telekinesis import constants, datatypes, retina
image = datatypes.Image.from_url(
url="https://assets.telekinesis.ai/examples/v1/images/warehouse_2.jpg"
)
detections = retina.detect_objects_using_yolox(
image=image,
score_threshold=0.80,
nms_threshold=0.45,
)
categories = constants.get_coco_categories(model="yolox")
for detection in detections:
label = categories[detection.category_id]
print(label.name, detection.score, detection.bbox)
The visualization below is the post-NMS result, not the network’s much larger dense candidate set. Compare it against crowded and overlapping scenes when selecting deployment thresholds.

Dense predictions become a short final list only after confidence filtering and suppression of overlapping candidates.
3. Benchmarking
3.1 Results Reported in the YOLOX Paper
The YOLOX paper reports both the final model family and the ablations that created it. That distinction matters: the family table tells you which size fits a device, while the ablation table explains where the accuracy came from.
Model Family: Accuracy, Compute, and Latency
Table 3 evaluates 640×640 inputs with FP16 and batch size 1 on a Tesla V100.
| Model | COCO AP | Parameters | GFLOPs | Latency |
|---|---|---|---|---|
| YOLOX-S | 39.6 | 9.0M | 26.8 | 9.8 ms |
| YOLOX-M | 46.4 | 25.3M | 73.8 | 12.3 ms |
| YOLOX-L | 50.0 | 54.2M | 155.6 | 14.5 ms |
| YOLOX-X | 51.2 | 99.1M | 281.9 | 17.3 ms |
The compact variants in Table 4 use 416×416 inputs:
| Model | COCO AP | Parameters | GFLOPs |
|---|---|---|---|
| YOLOv4-Tiny | 21.7 | 6.06M | 6.96 |
| YOLOX-Tiny | 32.8 | 5.06M | 6.45 |
| NanoDet | 23.5 | 0.95M | 1.20 |
| YOLOX-Nano | 25.3 | 0.91M | 1.08 |
COCO Test-Dev Accuracy Breakdown
Table 6 reports 300-epoch models at 640×640. APS, APM, and APL split performance by small, medium, and large objects.
| Model | FPS | AP | AP50 | AP75 | APS | APM | APL |
|---|---|---|---|---|---|---|---|
| YOLOX-DarkNet53 | 90.1 | 47.4 | 67.3 | 52.1 | 27.5 | 51.5 | 60.9 |
| YOLOX-M | 81.3 | 46.4 | 65.4 | 50.6 | 26.3 | 51.0 | 59.9 |
| YOLOX-L | 69.0 | 50.0 | 68.5 | 54.5 | 29.8 | 54.5 | 64.4 |
| YOLOX-X | 57.8 | 51.2 | 69.6 | 55.7 | 31.2 | 56.1 | 66.1 |
What the Training Choices Contributed
The paper reports that SimOTA raises the authors’ strong anchor-free baseline from 45.0 AP to 47.3 AP. Its augmentation study also shows that one recipe does not fit every model size:
| Model | Mosaic scale jitter | Extra augmentation | AP |
|---|---|---|---|
| YOLOX-Nano | [0.5, 1.5] | None | 25.3 |
| YOLOX-Nano | [0.1, 2.0] | MixUp | 24.0 |
| YOLOX-L | [0.1, 2.0] | None | 48.6 |
| YOLOX-L | [0.1, 2.0] | MixUp | 49.5 |
| YOLOX-L | [0.1, 2.0] | Copy-Paste | 49.4 |
The result is an engineering lesson: stronger augmentation helped the large model but hurt Nano, and latency depends on the exact hardware and implementation. Retina service performance additionally includes image transfer, decoding, preprocessing, model execution, NMS, serialization, and network latency, so benchmark the complete call with representative images.
3.2 Comparison with Other Telekinesis Skills
| Skill | Vocabulary | Post-processing control | Prefer it when |
|---|---|---|---|
| YOLOX | Fixed COCO-80 | Score + NMS thresholds | Dense real-time detection and NMS control |
| RF-DETR | Fixed COCO-80 | Score threshold | Transformer set prediction and global context |
| Grounding DINO | Open phrases | Box + text thresholds | Categories change at inference time |
| Qwen | Requested names | Object list | Broader semantic interpretation is needed |
3.3 YOLOX, RF-DETR, and Open-Vocabulary Detectors
YOLOX is a strong choice when the fixed COCO vocabulary fits, latency matters, and explicit NMS tuning is useful. RF-DETR offers transformer set prediction and stronger global interaction. Grounding DINO accepts changing text prompts, but prompt design and open-vocabulary calibration become new variables.
Benchmark them as complete pipelines. Model inference speed alone omits image transfer, decoding, preprocessing, post-processing, and network latency. Record median and tail latency, detection quality by scenario, and final robot task success.
4. Where to Go Next?
5. References
- Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. “YOLOX: Exceeding YOLO Series in 2021”, 2021.
- Redmon, J. et al. “You Only Look Once: Unified, Real-Time Object Detection”, CVPR, 2016.
- Lin, T.-Y. et al. “Feature Pyramid Networks for Object Detection”, CVPR, 2017.
- Bodla, N. et al. “Soft-NMS—Improving Object Detection With One Line of Code”, ICCV, 2017.


