Deep Neural Networks
Hands-on deep dives into the foundation models behind modern perception and robotics — from self-supervised vision backbones to open-vocabulary detectors, promptable segmentation, generative diffusion, large language models, and vision-language-action policies. Each page explains the ideas and measured results and links to the full runnable notebooks; interactive simulations build the intuition.
Grouped by what they produce. Every page is prose, figures and measured numbers — the code lives in the bundled notebooks linked from each page and from the notebook index at the bottom. The vision-language survey sits under this section too.
Object detection
From the set-prediction objective that removed NMS, through to an implementation you can read line by line with every tensor shape measured, and two production detectors built on the same idea.
DETR — set prediction
Detection as a bipartite matching problem: why one-to-one assignment makes NMS unnecessary, the Hungarian cost, the three loss terms, and deformable attention.
RF-DETR from scratch
Flat PyTorch, frozen DINOv2, the measured shape after every operation — plus the data bug that would have invalidated every result.
RF-DETR notebook
A real-time DETR on a DINOv2 backbone: deformable attention, Hungarian matching, no NMS.
GroundingDINO
Open-set detection: find any object named by free text via cross-modal image↔text fusion.
DINOv2
Self-supervised ViT features that transfer to classification, detection, segmentation and depth — with no labels.
Promptable segmentation
From boxes to pixels, and the accuracy/latency trade you make to get there in real time.
SAM
Segment Anything: a promptable mask model — click a point or draw a box, get a mask. Trained on 1.1B masks via a data engine.
FastSAM
SAM's "segment everything" at ~50× the speed by recasting it as YOLOv8 instance segmentation + prompt selection.
Vision-language models
The cross-model survey: what the families are, which modules they are assembled from, and the two tasks worth a deep dive on their own.
VLM survey
The research survey: families, modules, tasks and the open problems, sorted by compute tier.
Model comparison table
Sizes, training data, benchmark scores, architecture modules and editing capability, side by side.
Architecture modules
CLIP, Swin, FPN, connectors, flash attention, VAE, MMDiT — the parts everything is built from.
CLIP, SigLIP & BLIP
The contrastive objectives derived from scratch, and what each one fails at.
Visual question answering
Benchmarks, how the accuracy metric really works, language priors, hallucination.
Image editing & inpainting
Five mechanisms, inversion, and the locality trade that every editor makes.
Generative, language & action
Models that produce rather than label: images from text, text from text, and — at the end of the chain — robot actions from both.
Stable Diffusion deep dive
The math: forward/reverse SDE, the UNet, latent variables, cross-attention, samplers, rectified flow and MMDiT.
Stable Diffusion notebook
Latent diffusion: denoise in a VAE latent space guided by CLIP text, with classifier-free guidance.
LLMs & SQL-GRPO
Llama-2 architecture (RoPE, causal attention, SwiGLU) → QLoRA fine-tune → GRPO RL with an execution reward.
VLA & robotics
Vision-language-action models and task graphs — mapping perception + instruction straight to robot action.
OpenEMMA
A pretrained VLM asked to drive, run on real nuScenes frames — and the two diagnostics that show why its 8.75 m error is the less interesting problem.
The common thread: the transformer
Almost every model here is built from the same primitive — attention. A vision transformer tiles an image into patch tokens; a language model attends over word tokens; a detector's decoder attends from object queries to image features; a diffusion UNet uses cross-attention to inject text. Understanding attention once unlocks all of them:
And the vision models all start by turning an image into a sequence of patch tokens:
The notebooks behind these pages
Each model page ends with a code-free walkthrough extracted from its notebook series; the runnable originals are bundled with this site and open directly in Jupyter or VS Code.