Object detection & segmentation
How perception models find things and outline them: set-prediction detectors and the DINOv2 features behind them, open-vocabulary detection, promptable segmentation, and dense 3D occupancy from surround cameras — plus a full implementation walkthrough for BEVFusion and SurroundOcc.
2D detection
From the set-prediction objective that removed NMS to a real-time detector you can read line by line.
DETR — set prediction
Detection as bipartite matching: why one-to-one assignment makes NMS unnecessary, the Hungarian cost, the three loss terms, and deformable attention.
RF-DETR architecture
The real-time DETR block by block: DINOv2 backbone, multi-scale projector, deformable decoder and the training recipe.
RF-DETR from scratch
Flat PyTorch, frozen DINOv2, the measured shape after every operation — plus the data bug that would have invalidated every result.
RF-DETR notebook
A real-time DETR on a DINOv2 backbone: deformable attention, Hungarian matching, no NMS.
GroundingDINO
Open-set detection: find any object named by free text via cross-modal image↔text fusion.
DINOv2
Self-supervised ViT features that transfer to classification, detection, segmentation and depth — with no labels.
Segmentation
From boxes to pixels, and the accuracy/latency trade you make to get there in real time.
SAM
Segment Anything: a promptable mask model — click a point or draw a box, get a mask. Trained on 1.1B masks via a data engine.
FastSAM
SAM's "segment everything" at ~50× the speed by recasting it as YOLOv8 instance segmentation + prompt selection.
3D perception: BEV and occupancy
Lifting camera images into 3D — fused with LiDAR in a bird's-eye view, or filled into a dense semantic voxel grid.