A detector reports a point at pixel (800, 400), but a navigation system needs to know which direction that point lies in relative to the camera. The gap between those descriptions is the camera model: a compact set of assumptions that turns image measurements into geometric rays. I’ll start with what the sensor gives us, then work outward.
Pixels Are Not Angles
An image coordinate is measured on a pixel grid, while a ray direction is expressed in the camera’s coordinate frame. The intrinsic parameters connect the two: focal lengths fx and fy, measured in pixels, and the principal point (cx, cy), usually near the image center. These values belong to a calibrated camera model, not to the scene being photographed.
Turn A Pixel Into A Ray
For an ideal pinhole camera, convert pixel coordinates (u, v) to normalized image coordinates with x = (u – cx) / fx and y = (v – cy) / fy. For example, if fx is 800 pixels and cx is 640 pixels, u = 800 gives x = 0.2; the point is to the right of the optical axis. The corresponding camera-frame ray is proportional to (x, y, 1), then normalized if a unit direction is needed.
Check The Model Boundaries
Real lenses bend the ideal projection, especially toward the image edges, so a distorted pixel generally needs to be undistorted before applying the pinhole equations. Also check the calibration’s pixel-coordinate convention and camera-axis convention; changing the image origin or the direction assigned to the vertical axis changes the signs or offsets. A useful sanity check is to project the optical axis: it should land at the principal point, and its ray should point along the camera’s forward axis.