Algorithms
The estimation pages elsewhere on this site start from a motion model and ask what a filter can recover. This section starts from the other end: a pile of measurements with no labels — points, pixels, rays, scans — and asks how to extract geometry from them when a large fraction of the data is simply wrong. Robust fitting, multi-view geometry, the three structurally different ways to solve SLAM — and, for the learned parts of a modern stack, how to actually get training to converge.
Each page is a deep dive: the derivation, a symbol table, the failure modes with the numbers attached, an interactive simulation, and references to the original papers.
1 · Robust fitting & registration
Before any geometry can be estimated, the data has to be sorted into signal and noise — and in vision the contamination is not a few percent of mild outliers, it is routinely 50% of grossly wrong matches. Least squares has a breakdown point of zero: one bad datum can move the estimate arbitrarily far. These two pages are the standard answers.
RANSAC & robust estimation
Sample the minimal set, score by consensus, keep the champion. The trial-count formula, why it depends on the inlier fraction and not the data size, MSAC/LMedS/PROSAC, the threshold trade-off, and the degeneracy that makes a confident answer wrong.
ICP & point-cloud registration
The closed-form rotation from SVD, why point-to-plane converges in a fraction of the iterations, the local minimum that a bad initial guess falls into, trimming and robust kernels, and how GICP and NDT generalise the whole thing.
2 · Multi-view geometry
Cameras measure directions, not distances. The first page below establishes exactly what a camera does measure — the projection model, its parameters and how they are calibrated — and the rest are ways of recovering the missing range: from a second camera at a known baseline (stereo), from known 3-D structure (PnP), from motion (SfM), or from all of it at once in a single least-squares problem (bundle adjustment).
Camera model & projection
The equation every other page here depends on: world point → camera frame → perspective division → pixels. Intrinsics and what each parameter does, extrinsics and the sign convention that trips everyone, the Brown–Conrady distortion model, field of view, and why a calibration with low RMS can still be wrong.
Stereo & disparity
Rectification to a 1-D search, matching costs and the census transform, SGM's dynamic-programming smoothness, subpixel refinement, and the Z² law that governs every stereo error budget. Plus what learned matching changed and what it did not.
PnP — pose from correspondences
Why three points give up to four solutions, the P3P trigonometry, DLT and EPnP, iterative refinement of reprojection error, the two-fold ambiguity on planar targets, and how to tell a well-conditioned pose from a lucky one.
Structure from motion
The epipolar constraint, the 8-point and 5-point algorithms, the four-way decomposition of the essential matrix, triangulation and its parallax dependence, scale drift, and the incremental vs global pipeline trade.
Bundle adjustment
The maximum-likelihood problem at the end of every pipeline. Levenberg–Marquardt, the primary sparsity structure, the Schur complement that makes it tractable, gauge freedom, robustification, and why the covariance it reports is conditional.
3D Gaussian splatting
Novel-view synthesis at 100+ FPS by changing the representation rather than the optimiser. The covariance factorisation, EWA projection through the same Jacobian PnP uses, the tiled sorted rasteriser, spherical-harmonic colour, and how adaptive densification trains it.
3 · Training deep models
Everything above is derived from geometry, so it either works or it does not. Learned components fail differently: the architecture is usually fine and the training is misconfigured. This page is the practical counterpart to the rest of the section.
Hyperparameter tuning & training
Which knobs matter and in what order; the learning rate and its coupling to batch size; optimizers and why the SGD/Adam LR gap is 100×; choosing losses for detection, segmentation and contrastive training; per-architecture recipes for YOLO, ViT, Faster/Mask R-CNN and CLIP; getting the PyTorch loop right; and a diagnostic ladder for a run that will not converge.
DNN optimizers
Every optimizer in use is a cheap stand-in for second-order information. The lineage from SGD through momentum, Adagrad and RMSProp to Adam and AdamW, why Adam's step is scale-invariant and bounded, what convergence theory does and does not guarantee, the memory cost of optimizer state at LLM scale, and when it is worth leaving the default.
4 · Motion planning
Planning in more than a few dimensions defeats grid search, because the number of cells grows exponentially with the degrees of freedom. The sampling-based family avoids building a grid at all — and the connection to RANSAC is direct: both replace an intractable search with randomised sampling plus a cheap feasibility test.
RRT & RRT* planning
The Voronoi bias that makes a random tree explore outward without tracking a frontier; probabilistic completeness; why plain RRT converges to the optimum with probability zero; and the rewiring plus shrinking connection radius that make RRT* asymptotically optimal.
Two practical defaults
RRT-Connect when you need a feasible path fast — bidirectional and greedy. Informed RRT* whenever you want quality: once a solution exists, restricting samples to the ellipsoid that could improve it is a strict win with no downside.
5 · SLAM
Simultaneous localisation and mapping is one problem with three classic solutions, and they differ in exactly one design decision: what to do about the correlations between the robot pose and the map. Keep them all in one covariance (EKF-SLAM), sample the pose so the map factorises (FastSLAM), or keep the raw constraints and optimise the whole history at once (GraphSLAM). Read them in that order — each one is a reaction to the previous one's cost.
EKF-SLAM
One state vector, one covariance, full correlation. Landmark initialisation, the O(n²) update, data association and gating, and the linearisation that makes it inconsistent.
FastSLAM
Rao-Blackwellisation: sample the trajectory, and the landmarks become conditionally independent. FastSLAM 1.0 vs 2.0, and the particle depletion that limits it.
GraphSLAM & pose graphs
MAP estimation on a graph of constraints. The information form, SE(2)/SE(3) Jacobians, loop closure, and the robust kernels that survive a false match.
6 · How the pieces fit together
These are not nine unrelated topics. A visual SLAM system uses almost all of them in one pass, and the order is always roughly the same: detect and match, reject outliers, get a pose, triangulate structure, refine everything jointly, and close the loop.
Read that diagram as a statement about time budgets. The front end must produce a pose every frame, so it uses closed-form solvers and a bounded number of RANSAC trials. The back end is allowed to be slow and iterative, because it is improving an estimate that already exists. Almost every practical system is organised on this split.
7 · Still to come
This section is being built out from a wider inventory. The pages above are complete; the topics below are planned, and several already have partial coverage elsewhere on the site that the eventual page will consolidate rather than duplicate.
| Topic | Why it belongs here | Existing coverage |
|---|---|---|
| Multi-hypothesis tracking | Data association as a hypothesis tree rather than a single nearest-neighbour choice; the counterpart to RANSAC for time-series association | Perception stack (greedy + Hungarian) |
| 2-D detection: YOLO, Faster R-CNN | Anchors, one-stage vs two-stage, the loss that makes detection work | RF-DETR, NMS sim |
| 3-D detection: PointNet, PointPillars | Permutation invariance, pillar encoding, the BEV bridge to fusion | BEVFusion |
| Transformers: ViT, Swin, CLIP | Patchification, shifted-window attention, contrastive alignment | DINOv2, attention sims |
| RRT / RRT* | Sampling-based planning; probabilistic completeness and the rewiring that makes it optimal | — |
| Inverse kinematics | Jacobian pseudo-inverse, damped least squares, singularity handling, null-space motion | — |
| PID & loop shaping | The controller everything actually ships with, and how to tune it honestly | GNC (step response, LQR sims) |
| GNSS receiver internals | Acquisition search, tracking loops, PVT — the layer below a pseudorange | GNSS, receivers |
| Conjugate Unscented Transform | Higher-order moment matching with far fewer points than a full cubature rule | Filtering (UKF sigma points) |
| Method of characteristics | Propagating a density along its own flow instead of a covariance approximation | — |
Related sections: GNC & Estimation · Perception stack · Deep Learning · Techniques