Model size, throughput and the hardware it runs on

1 · The hardware, and why the same model changes speed on it

A perception model's frame rate is set as much by the silicon as by the network. Three properties dominate: tensor-core throughput at the precision you actually deploy, memory bandwidth (BEV and voxel workloads move large feature maps), and the power/thermal envelope, which on an embedded module is the difference between a datasheet number and a sustained one.

1.1 Data-center and desktop GPUs

1.2 NVIDIA Jetson modules (the actual deployment targets)

Specifications as published in the Ultralytics Jetson guide's comparison table:

2 · 2D detectors: parameters, FLOPs and accuracy

Parameter counts, multiply-accumulate cost and COCO box mAP straight from the torchvision pretrained-weight metadata (num_params, _ops, _file_size, _metrics). These are the reference implementations of Faster R-CNN, Mask R-CNN and SSD, all evaluated on COCO val2017.

Source: torchvision detection weight metadata. GFLOPs there are for the model's default input transform, and these two-stage models are resolution-dependent — cost scales with the resized image, so a single GFLOPs figure is not a fixed budget. torchvision publishes no per-GPU latency, so there is no honest A100 or Jetson FPS column for these rows; the ratio of GFLOPs to YOLO's is the only comparison available, and it is a weak proxy.

3 · YOLO: the one family with device-by-device published latency

3.1 YOLOv8 on A100, with CPU baseline

COCO val, 640 px input, TensorRT on A100 and ONNX on CPU:

Source: Ultralytics YOLOv8 model card. The FPS column is my arithmetic 1/latency and is single-stream inference only: it excludes decode, letterboxing, NMS and any host transfer, so a deployed pipeline will be slower. Ultralytics does not state batch size for the A100 column. Note YOLOv8 is compared on A100 while YOLO11 is compared on T4 — those two tables cannot be read against each other.

3.2 YOLO26 across four Jetson modules and three precisions

This is the most useful table on the page: one model family, one input size (640), one benchmark run, four devices, FP32/FP16/INT8. Latency is ms per image; engine size is on-disk MB; mAP is COCO mAP_50-95 as measured by Ultralytics after export.

Source: Ultralytics NVIDIA Jetson guide, parsed with tools/parse_jetson_tables.py so each latency keeps its device and model attribution. Thor/Orin tables were run on JetPack 7.2, Orin Nano on JP6.1, Orin NX on JP6.0/JP5.1.3 — different JetPack and TensorRT versions per column. Engine size and post-export mAP vary slightly per device, so a range is shown. Ultralytics' table publishes no FP32 row for Orin NX; the cell is left empty rather than interpolated. One upstream value is clearly a typo (AGX Orin YOLO26n FP16 mAP is listed as 0.0450 where every other device gives ≈0.479) and is excluded from the range shown.

4 · LiDAR and BEV models

4.1 What the original papers claimed, and on what

Sources: PointPillars (the 62/105 Hz and the 225 ms VoxelNet comparison), BEVFusion, PointNet / PointNet++. I checked mmdetection3d's PointPillars model zoo for per-config throughput: its Inf time (fps) column is empty for every row, so no FPS can be cited from there. It does publish accuracy and training memory (e.g. nuScenes SECFPN mAP 34.33 / NDS 49.1 at 16.4 GB; the FP16 variant mAP 35.19 / NDS 50.27 at 8.37 GB), which is a useful note that mixed precision here changed memory and accuracy, not a published frame rate.

4.2 BEVFusion optimized for Orin — the quantization story end to end

Source: NVIDIA CUDA-BEVFusion, measured with TensorRT 8.6 / CUDA 11.4 / cuDNN 8.6 and averaged over the 6019 nuScenes validation samples. NVIDIA notes the figure is sensitive to LiDAR point count, so scene density moves it.

5 · DINOv2 and transformer backbones

Source: DINOv2 repository model table (the with-registers variants have identical parameter counts and slightly different scores). DINOv2 publishes no inference latency for any GPU — the repository's A100 references are to training on 4–12 nodes of A100-80GB, which is not an inference benchmark. Any "DINOv2 FPS" figure would therefore have to be measured locally, and at ViT-L/ViT-g sizes it is a feature extractor you budget per-frame, not a real-time detector.

For deployed detection the relevant derived model on this site is RF-DETR, which puts a DINOv2-Small backbone in front of an RT-DETR decoder. Its notebook series quotes ~29 M parameters at 53.3 COCO mAP and ~60 FPS on A100, with RF-DETR-large at ~128 M and 56.5 mAP. Those figures come from the tutorial's own comparison table rather than from a first-party Roboflow benchmark page, so treat them as indicative — and as the same kind of single-stream number as the YOLO column.

6 · Segmentation on older Jetson hardware

The only published Nano-class numbers I could source are NVIDIA's jetson-inference FCN-ResNet18 segmentation benchmarks — useful because they show how hard resolution hits a small module. FP16, JetPack 4.2.1, nvpmodel 0 (MAX-N):

Source: jetson-inference. For a rough sense of peak small-model throughput on Volta-class hardware, NVIDIA's separate jetson_benchmarks sample output for Xavier NX lists ssd-mobilenet-v1 at 887.6 FPS and ResNet50 224×224 at 824.02 FPS — those are TensorRT throughput benchmarks with batching, not per-frame pipeline latency.

7 · Reproducing and extending these tables

Two scripts in tools/ fetch and attribute the figures, so the page can be re-checked when upstream changes:

python3 tools/fetch_model_specs.py      # saves filtered primary sources to /tmp/specs/
python3 tools/parse_jetson_tables.py   # device|model|format|size|mAP|ms rows
python3 tools/fetch_backbone_specs.py  # DINOv2 param table, OpenPCDet metrics

To measure your own model rather than cite someone else's, the minimum honest protocol is:

1. Fix and report device, power mode, clocks, JetPack/TensorRT/CUDA versions, input size, batch size and precision. On Jetson, set nvpmodel and jetson_clocks explicitly, and say which you used.
2. Build the engine on the target device, and calibrate INT8 there with representative data.
3. Warm up, then report a latency distribution (p50/p99/p99.9), not a mean — and measure sustained throughput long enough to hit thermal steady state, because an embedded module throttles.
4. Report end-to-end age (capture → result), not just the inference kernel: decode, H2D copy, preprocessing and NMS are part of the vehicle's reaction time.
5. Re-measure accuracy after every optimization, on the same split, and publish the delta. A speed number without its accuracy cost is not a result.