FastSAM — SAM's speed problem, solved

FastSAM (CASIA-IVA-Lab, 2023) reproduces SAM's "segment everything" at roughly 50× the speed by recasting it as a familiar, efficient task: class-agnostic instance segmentation with YOLOv8-seg, followed by a prompt-selection stage. It trades a little quality for real-time throughput.

Symbols used on this page are defined in the Nomenclature → deep learning & training.

FastSAM paper (arXiv:2306.12156) 5-notebook deep dive in deeplearning/notebooks/fastSAM/

The YOLOv8-seg pipeline

image (B,3,1024,1024)
  → Backbone (CSP/C2f): P3(/8), P4(/16), P5(/32)
  → Neck (PAN-FPN): multi-scale fusion
  → Segment head:  box(DFL 4·16) · cls(1) · mask-coeff(32) per anchor
  → Proto branch:  k=32 prototype masks at input/4 (256×256)

The clever part is prototype masks: instead of predicting a full mask per instance, the model learns 32 shared prototype masks and, per detection, a 32-vector of coefficients. The instance mask is a linear combination passed through a sigmoid, then cropped to the box:

$$ \text{mask}_i = \sigma\!\Big(\underbrace{\mathbf{c}_i}_{(1\times32)}\;\underbrace{P}_{(32\times HW)}\Big), \qquad P=\text{prototypes} $$

Training losses

YOLOv8-seg is trained with a TaskAlignedAssigner and four losses:

CIoU box regression + DFL (distribution focal loss) for precise box edges,
BCE for objectness/class,
mask BCE on the assembled prototype masks.

FastSAM is trained on just 2% of SA-1B, yet recovers most of SAM's everything-mode quality.

Prompting after the fact

FastSAM first segments everything, then selects masks matching your prompt: a point keeps masks containing it, a box keeps masks with high IoU, and text uses CLIP to score each mask crop against the phrase. The heavy lifting (segmentation) is done once; prompting is cheap post-processing.

Under the hood: it's a detector

Because FastSAM is YOLO-based, it inherits the anchor/NMS machinery of classical detectors — many raw candidate masks are pruned by non-max suppression before prompt selection:

Variant	Backbone	Params	Size
FastSAM-s	YOLOv8s-seg	~11.8M	23 MB
FastSAM-x	YOLOv8x-seg	~68M	138 MB

SAM vs FastSAM. SAM is a promptable transformer optimized for mask quality and zero-shot flexibility; FastSAM is a real-time detector optimized for throughput. Choose SAM for interactive precision, FastSAM for video-rate "segment everything."

Full notebook deep-dive

The complete, code-free explanations, math, and figures extracted from the model's Jupyter notebooks. The runnable notebooks are bundled with this site — browse the FastSAM notebooks (local copies).

FastSAM 01 - Overview & Architecture

FastSAM (Fast Segment Anything, Zhao et al., CASIA-IVA-Lab, 2023) achieves SAM-style 'segment everything' at a fraction of the cost by decoupling the task into two stages:

All-instance segmentation with a YOLOv8-seg CNN detector (trained on just 2% of SA-1B), producing class-agnostic masks for every object.
Prompt-guided selection: given a point / box / text prompt, pick the matching mask(s) from stage 1.

The result runs ~50x faster than SAM's ViT-H image encoder while reaching comparable zero-shot mask quality, because a CNN at 640-1024 px is dramatically cheaper than a heavy ViT.

FastSAM vs SAM at a glance

	SAM (ViT-H)	FastSAM (YOLOv8x-seg)
Backbone	ViT-H (632M params)	CSP-Darknet / C2f (~68M)
Paradigm	promptable transformer	detect-then-select CNN
Masks	decoder per prompt	prototype masks + per-instance coeffs
'Everything' speed	slow (dense grid of prompts)	single forward pass
Training data	full SA-1B (1.1B masks)	2% of SA-1B

FastSAM comes in two sizes: FastSAM-s (YOLOv8s-seg, ~11.8M params, 23MB) and FastSAM-x (YOLOv8x-seg, ~68M params, 138MB).

The YOLOv8-seg pipeline

The prototype-mask trick (from YOLACT) is the key: instead of a full per-instance mask branch, the head predicts k=32 shared prototype masks once, plus 32 mask coefficients per detected instance. A linear combination + sigmoid yields each instance mask - very cheap.

Load the real FastSAM model and inspect it

Loaded FastSAM-s.pt
task           : segment
num parameters : 11,790,483
num classes    : 1 (class-agnostic: 1 -> object)
top module types: {'Conv2d': 76, 'Conv': 66, 'BatchNorm2d': 66, 'ModuleList': 11, 'Sequential': 10, 'Bottleneck': 10, 'C2f': 8, 'Concat': 4, 'Upsample': 2, 'SegmentationModel': 1}

The prototype-mask assembly, illustrated

We synthesize k=32 prototypes and a set of instance coefficients to show exactly how the final masks are formed - the same math the real head uses.

protos : (32, 160, 160) (k, Hp, Wp)
coeffs : (3, 32) (N_instances, k)
masks  : (3, 160, 160) = sigmoid(coeffs @ protos)

saved 01_prototype_masks.png

Next: NB02 traces the exact tensor shape at every stage of the network on a real image.

FastSAM 02 - Architecture Deep-Dive: Tensor Sizes & Features

We push a real image through FastSAM and record the tensor shape at each stage: preprocessing, the three backbone scales (P3/P4/P5), the neck, and the segment head outputs (boxes, mask coefficients, prototypes). Then we visualize backbone feature maps and the raw prototype masks.

Preprocessing

Ultralytics letterboxes the image to a multiple of the stride (32) - commonly 1024x1024 for FastSAM - normalizes to [0,1], and forms pixel_values (B,3,1024,1024).

raw image : (480, 640, 3) from /mnt/mydata/huggingface/sample_coco.jpg
letterboxed input target: (1, 3, 1024, 1024)
strides -> feature grids:
  P3  stride  8 -> 128x128 grid
  P4  stride 16 -> 64x64 grid
  P5  stride 32 -> 32x32 grid

Hook the real network and record shapes

We register forward hooks on the backbone/neck/head modules and read off the tensor shapes produced for our image. This is the ground truth for FastSAM-s.

layer | type | output shape
   0 | Conv           | (1, 32, 512, 512)
   1 | Conv           | (1, 64, 256, 256)
   2 | C2f            | (1, 64, 256, 256)
   3 | Conv           | (1, 128, 128, 128)
   4 | C2f            | (1, 128, 128, 128)
   5 | Conv           | (1, 256, 64, 64)
   6 | C2f            | (1, 256, 64, 64)
   7 | Conv           | (1, 512, 32, 32)
   8 | C2f            | (1, 512, 32, 32)
   9 | SPPF           | (1, 512, 32, 32)
  10 | Upsample       | (1, 512, 64, 64)
  11 | Concat         | (1, 768, 64, 64)
  12 | C2f            | (1, 256, 64, 64)
  13 | Upsample       | (1, 256, 128, 128)
  14 | Concat         | (1, 384, 128, 128)
  15 | C2f            | (1, 128, 128, 128)
  16 | Conv           | (1, 128, 64, 64)
  17 | Concat         | (1, 384, 64, 64)
  18 | C2f            | (1, 256, 64, 64)
  19 | Conv           | (1, 256, 32, 32)
  20 | Concat         | (1, 768, 32, 32)
  21 | C2f            | (1, 512, 32, 32)
  22 | Segment        | ['tuple', 'dict']

The segment-head output shapes

Output	Shape (FastSAM-s @1024)	Meaning
`protos`	`(1, 32, 256, 256)`	32 prototype masks at input/4
box (DFL)	`(1, 64, A)`	4 sides x reg_max(16) distributions
cls	`(1, 1, A)`	single 'object' class logit
mask coeffs	`(1, 32, A)`	32 proto coeffs per anchor

A = total anchors summed over the 3 FPN levels (128^2 + 64^2 + 32^2 = 21504 at 1024). After NMS, only the kept N instances survive; each instance mask = sigmoid(coeffs_N x protos) cropped to its box.

Visualize backbone feature maps + prototypes on the real image

detections: 11

saved 02_feature_maps.png

Next: NB03 covers how FastSAM is trained and the exact loss functions.

FastSAM 03 - Training Approach & Loss Functions

FastSAM is trained exactly like a YOLOv8 instance-segmentation model, but on a class-agnostic version of SA-1B (only 2% of it, ~46k images). Every SA-1B mask becomes a single-class ('object') instance with a tight bounding box + polygon mask. The model then learns to detect + segment all objects in one pass.

Inputs and targets

Input: letterboxed image (B,3,1024,1024).
Targets (per image): a list of instances, each with a box (cx,cy,w,h) normalized, class 0, and a binary mask (stored as a polygon in the YOLO-seg label .txt).
Assignment: YOLOv8 uses TaskAlignedAssigner to match anchors to GT by a combined score s = p^alpha * IoU^beta (classification confidence x localization quality), replacing hand-tuned anchor matching.

The four loss terms

YOLOv8-seg total loss is a weighted sum:

$$\mathcal{L} = \lambda_{box}\mathcal{L}_{CIoU} + \lambda_{cls}\mathcal{L}_{BCE} + \lambda_{dfl}\mathcal{L}_{DFL} + \lambda_{seg}\mathcal{L}_{mask}$$

Box - CIoU loss on matched anchors (complete IoU: overlap + center distance + aspect ratio).
Classification - BCE on the objectness/class logit.
DFL (Distribution Focal Loss) - the box regression is a distribution over reg_max=16 bins per side; DFL sharpens it toward the continuous target.
Mask loss - BCE between the assembled instance mask sigmoid(coeffs @ protos) (cropped to the GT box) and the GT mask, averaged over matched instances.

Default gains (ultralytics): box=7.5, cls=0.5, dfl=1.5, seg mask gain within the box term.

CIoU(pred,gt) = 0.7808 -> loss = 0.2192
DFL loss example = 3.7718
mask BCE = 0.1389

Visualize the DFL idea

Each box side is predicted as a softmax distribution over 16 bins; the final coordinate is the distribution's expectation. DFL pushes probability mass onto the two bins straddling the true value.

saved 03_dfl.png

Training recipe (FastSAM defaults)

Optimizer: SGD/AdamW, cosine LR, warmup, EMA of weights.
Heavy augmentation: mosaic, mixup, HSV jitter, random flip/scale (mosaic disabled for the last ~10 epochs).
100 epochs on the 2% SA-1B subset at imgsz=1024.
Because there is 1 class, the model is purely a region/instance proposer - semantic labels come later from the prompt stage.

Next: NB04 runs real inference in everything/box/point/text modes; NB05 fine-tunes on aerial imagery.

FastSAM 04 - Inference (Everything / Box / Point / Text)

FastSAM's ultralytics API runs stage-1 all-instance segmentation once, then the FastSAMPrompt helper selects masks by prompt. We demonstrate all four modes on a real image.

image: (480, 640, 3) source: /mnt/mydata/huggingface/sample_coco.jpg

FastSAM ready; stage-1 instances: 11

Mode 1 - 'Everything' (all instances in one pass)

saved 04_everything.png

Mode 2 - Box prompt

FastSAMPrompt.box_prompt keeps the stage-1 mask with highest IoU to the given box.

saved 04_box_prompt.png

Mode 3 - Point prompt

saved 04_point_prompt.png

Mode 4 - Text prompt (CLIP-guided)

text_prompt embeds the stage-1 mask crops and the text with CLIP and keeps the best-matching mask. Requires the clip package + a CLIP checkpoint; if unavailable we note it and skip gracefully.

saved 04_text_prompt.png; text_prompt_ran = False

Next: NB05 fine-tunes FastSAM on the harshinde/Aerial-Imagery dataset.

FastSAM 05 - Fine-Tuning on Aerial Imagery

We fine-tune FastSAM (a YOLOv8-seg model) on harshinde/Aerial-Imagery, a semantic-segmentation dataset of drone/aerial tiles with per-pixel masks. Because FastSAM trains via the YOLOv8-seg pipeline, we must:

Download the dataset (image + PNG mask pairs) from the HF Hub.
Convert each semantic mask into YOLO-seg polygon labels (class x1 y1 x2 y2 ... normalized).
Write an ultralytics data.yaml and train with model.train(...).

The dataset layout is Tile N/images/.jpg + Tile N/masks/.png. Masks are color/greyscale-coded classes; we treat each connected component of a foreground class as one instance.

Step 1 - Download the dataset from the HF Hub

We snapshot the repo into /mnt/mydata/huggingface/aerial_imagery. Uses HF_TOKEN if present; skips gracefully (schematic synthetic tiles) if offline.

downloaded to /mnt/mydata/huggingface/aerial_imagery
image dirs found: 8
AERIAL_DIR = /mnt/mydata/huggingface/aerial_imagery

Step 2 - Convert semantic masks to YOLO-seg polygon labels

For each mask PNG we threshold to foreground, find connected components (via cv2.findContours), and write one polygon line per component: class_id x1 y1 x2 y2 ... with coordinates normalized to [0,1]. Tiny components are dropped. Images+labels are split train/val and laid out in the ultralytics folder convention.

image/mask pairs: 72

per-split (images, polygons): {'val': (14, 106), 'train': (58, 307)}

Visualize a converted sample (image + polygons)

saved 05_label_sample.png

Step 3 - Write `data.yaml` and fine-tune

One class (object). We start from FastSAM-s.pt and run a short fine-tune (few epochs, small imgsz) so the notebook completes quickly; bump epochs/imgsz for real training. Ultralytics writes runs under FASTSAM_RUNS_DIR.

path: /mnt/mydata/huggingface/aerial_yolo
train: images/train
val: images/val
names:
  0: object

Ultralytics 8.4.115 🚀 Python-3.10.12 torch-2.3.0+cu121 CUDA:0 (NVIDIA RTX 6000 Ada Generation, 48639MiB)

engine/trainer: agnostic_nms=False, amp=True, angle=1.0, augment=False, auto_augment=randaugment, batch=4, bgr=0.0, box=7.5, cache=False, cfg=None, channels_last=False, classes=None, close_mosaic=10, cls=0.5, cls_pw=0.0, cls_remap=True, compile=False, conf=None, copy_paste=0.0, copy_paste_mode=flip, cos_lr=False, cutmix=0.0, data=/mnt/mydata/huggingface/aerial_yolo/data.yaml, degrees=0.0, deterministic=True, device=0, dfl=1.5, dgrad=0.5, dis=6.0, distill_model=None, dlam=1.0, dlog=1.0, dnn=False, dropout=0.0, dynamic=False, embed=None, end2end=None, epochs=2, erasing=0.4, exist_ok=True, fliplr=0.5, flipud=0.0, format=torchscript, fraction=1.0, freeze=None, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, imgsz=256, iou=0.7, keras=False, kobj=1.0, line_width=None, lr0=0.01, lrf=0.01, mask_ratio=4, max_det=300, mixup=0.0, mode=train, model=/mnt/mydata/huggingface/fastsam/FastSAM-s.pt, momentum=0.937, mosaic=1.0, multi_scale=0.0, name=aerial_ft, nbs=64, nms=False, opset=None, optimize=False, optimizer=auto, overlap_mask=True, patience=100, perspective=0.0, plots=False, pose=12.0, pretrained=True, profile=False, project=/mnt/mydata/huggingface/fastsam_runs, quantize=None, rect=False, resume=False, retina_masks=False, rle=1.0, save=True, save_conf=False, save_crop=False, save_dir=/mnt/mydata/huggingface/fastsam_runs/aerial_ft, save_frames=False, save_json=False, save_period=-1, save_txt=False, scale=0.5, seed=0, shear=0.0, show=False, show_boxes=True, show_conf=True, show_labels=True, simplify=True, single_cls=False, source=None, split=val, stream_buffer=False, task=segment, time=None, tracker=tracktrack.yaml, translate=0.1, val=True, verbose=False, vid_stride=1, visualize=False, warmup_bias_lr=0.1, warmup_epochs=3.0, warmup_momentum=0.8, weight_decay=0.0005, workers=2, workspace=None

Downloading https://ultralytics.com/assets/Arial.ttf to '/tmp/Ultralytics/Arial.ttf': 77% ━━━━━━━━━─── 584.0/755.1KB 5.7MB/s 0.1s<0.0s

Downloading https://ultralytics.com/assets/Arial.ttf to '/tmp/Ultralytics/Arial.ttf': 100% ━━━━━━━━━━━━ 755.1KB 6.0MB/s 0.1s

                   from n params module arguments

  0                  -1  1       928  ultralytics.nn.modules.conv.Conv             [3, 32, 3, 2]

  1                  -1  1     18560  ultralytics.nn.modules.conv.Conv             [32, 64, 3, 2]

  2                  -1  1     29056  ultralytics.nn.modules.block.C2f             [64, 64, 1, True]

  3                  -1  1     73984  ultralytics.nn.modules.conv.Conv             [64, 128, 3, 2]

  4                  -1  2    197632  ultralytics.nn.modules.block.C2f             [128, 128, 2, True]

  5                  -1  1    295424  ultralytics.nn.modules.conv.Conv             [128, 256, 3, 2]

  6                  -1  2    788480  ultralytics.nn.modules.block.C2f             [256, 256, 2, True]

  7                  -1  1   1180672  ultralytics.nn.modules.conv.Conv             [256, 512, 3, 2]

  8                  -1  1   1838080  ultralytics.nn.modules.block.C2f             [512, 512, 1, True]

  9                  -1  1    656896  ultralytics.nn.modules.block.SPPF            [512, 512, 5]

 10                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']

 11             [-1, 6]  1         0  ultralytics.nn.modules.conv.Concat           [1]

 12                  -1  1    591360  ultralytics.nn.modules.block.C2f             [768, 256, 1]

 13                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']

 14             [-1, 4]  1         0  ultralytics.nn.modules.conv.Concat           [1]

 15                  -1  1    148224  ultralytics.nn.modules.block.C2f             [384, 128, 1]

 16                  -1  1    147712  ultralytics.nn.modules.conv.Conv             [128, 128, 3, 2]

 17            [-1, 12]  1         0  ultralytics.nn.modules.conv.Concat           [1]

 18                  -1  1    493056  ultralytics.nn.modules.block.C2f             [384, 256, 1]

 19                  -1  1    590336  ultralytics.nn.modules.conv.Conv             [256, 256, 3, 2]

 20             [-1, 9]  1         0  ultralytics.nn.modules.conv.Concat           [1]

 21                  -1  1   1969152  ultralytics.nn.modules.block.C2f             [768, 512, 1]

 22        [15, 18, 21]  1   2770931  ultralytics.nn.modules.head.Segment          [1, 32, 128, 16, None, [128, 256, 512]]

YOLOv8s-seg summary: 152 layers, 11,790,483 parameters, 11,790,467 gradients, 40.2 GFLOPs

Transferred 417/417 items from pretrained weights

Freezing layer 'model.22.dfl.conv.weight'

AMP: running Automatic Mixed Precision (AMP) checks...

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 7% ╸─────────── 416.0KB/5.3MB 4.0MB/s 0.1s<1.2s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 21% ━━╸───────── 1.1/5.3MB 7.3MB/s 0.2s<0.6s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 35% ━━━━──────── 1.9/5.3MB 7.5MB/s 0.3s<0.5s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 48% ━━━━━╸────── 2.6/5.3MB 6.9MB/s 0.4s<0.4s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 57% ━━━━━━╸───── 3.1/5.3MB 4.4MB/s 0.5s<0.5s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 72% ━━━━━━━━╸─── 3.8/5.3MB 7.1MB/s 0.6s<0.2s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 85% ━━━━━━━━━━── 4.5/5.3MB 6.7MB/s 0.7s<0.1s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 93% ━━━━━━━━━━━─ 5.0/5.3MB 4.5MB/s 0.8s<0.1s

Downloading https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26n.pt to 'yolo26n.pt': 100% ━━━━━━━━━━━━ 5.3MB 6.0MB/s 0.9s

AMP: checks passed ✅

train: Fast image access ✅ (ping: 0.0±0.0 ms, read: 3166.4±1040.4 MB/s, size: 150.1 KB)

train: Scanning /mnt/mydata/huggingface/aerial_yolo/labels/train... 58 images, 1 backgrounds, 0 corrupt: 100% ━━━━━━━━━━━━ 58/58 1.6Kit/s 0.0s

train: New cache created: /mnt/mydata/huggingface/aerial_yolo/labels/train.cache

val: Fast image access ✅ (ping: 0.0±0.0 ms, read: 2369.4±1383.3 MB/s, size: 665.1 KB)

val: Scanning /mnt/mydata/huggingface/aerial_yolo/labels/val... 14 images, 0 backgrounds, 0 corrupt: 100% ━━━━━━━━━━━━ 14/14 1.3Kit/s 0.0s

val: New cache created: /mnt/mydata/huggingface/aerial_yolo/labels/val.cache

optimizer: 'optimizer=auto' found, ignoring 'lr0=0.01' and 'momentum=0.937' and determining best 'optimizer', 'lr0' and 'momentum' automatically...

optimizer: AdamW(lr=0.002, momentum=0.9) with parameter groups 66 weight(decay=0.0), 77 weight(decay=0.0005), 76 bias(decay=0.0)

Image sizes 256 train, 256 val
Using 2 dataloader workers
Logging results to /mnt/mydata/huggingface/fastsam_runs/aerial_ft
Starting training for 2 epochs...

      Epoch    GPU_mem box_loss seg_loss cls_loss dfl_loss sem_loss  Instances       Size

        1/2     0.574G     0.8909      5.245      1.502      1.168          0         33        256: 0% ──────────── 0/15  1.5s

        1/2     0.592G      1.077      6.167      1.904      1.241          0         33        256: 13% ━╸────────── 2/15 4.4it/s 1.7s<2.9s

        1/2     0.592G      1.162      6.572      1.962      1.301          0         25        256: 26% ━━━───────── 4/15 8.3it/s 1.8s<1.3s

        1/2     0.592G      1.346      7.064      2.011      1.422          0         37        256: 40% ━━━━╸─────── 6/15 10.3it/s 1.9s<0.9s

        1/2     0.592G      1.379      6.798      1.939      1.413          0         61        256: 53% ━━━━━━────── 8/15 12.6it/s 2.0s<0.6s

        1/2     0.592G       1.37      6.916      1.921       1.41          0         41        256: 66% ━━━━━━━━──── 10/15 12.9it/s 2.2s<0.4s

        1/2     0.592G      1.388      6.766      1.914      1.404          0         50        256: 80% ━━━━━━━━━╸── 12/15 14.2it/s 2.3s<0.2s

        1/2     0.592G      1.394      6.907      1.961      1.414          0         11        256: 93% ━━━━━━━━━━━─ 14/15 10.9it/s 2.9s<0.1s

        1/2     0.592G      1.394      6.907      1.961      1.414          0         11        256: 100% ━━━━━━━━━━━━ 15/15 5.2it/s 2.9s

                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95)     Mask(P          R      mAP50  mAP50-95): 50% ━━━━━━────── 1/2 1.0it/s 0.3s<1.0s

                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95)     Mask(P          R      mAP50  mAP50-95): 100% ━━━━━━━━━━━━ 2/2 3.9it/s 0.5s

                   all         14        106      0.194      0.292      0.138     0.0827      0.189      0.264     0.0896     0.0427

      Epoch    GPU_mem box_loss seg_loss cls_loss dfl_loss sem_loss  Instances       Size

        2/2     0.592G      1.298      6.457      1.882      1.323          0         31        256: 6% ╸─────────── 1/15 2.6it/s 0.1s<5.5s

        2/2     0.592G      1.358      7.375      1.857      1.351          0         34        256: 20% ━━────────── 3/15 6.3it/s 0.3s<1.9s

        2/2     0.592G      1.441      7.283      1.984       1.37          0         31        256: 33% ━━━━──────── 5/15 9.3it/s 0.4s<1.1s

        2/2     0.592G      1.472      7.005      2.047      1.414          0         37        256: 46% ━━━━━╸────── 7/15 10.7it/s 0.5s<0.7s

        2/2     0.592G       1.48      7.152      2.056      1.425          0         44        256: 60% ━━━━━━━───── 9/15 12.2it/s 0.6s<0.5s

        2/2     0.592G      1.473      7.125      2.043      1.426          0         45        256: 73% ━━━━━━━━╸─── 11/15 12.8it/s 0.8s<0.3s

        2/2     0.592G      1.447      7.042      2.038       1.42          0         31        256: 86% ━━━━━━━━━━── 13/15 13.6it/s 0.9s<0.1s

        2/2     0.592G      1.473      6.954       2.03      1.425          0         38        256: 100% ━━━━━━━━━━━━ 15/15 15.5it/s 1.0s

                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95)     Mask(P          R      mAP50  mAP50-95): 100% ━━━━━━━━━━━━ 2/2 18.5it/s 0.1s

                   all         14        106      0.234      0.368      0.164        0.1      0.191      0.292     0.0979     0.0474

2 epochs completed in 0.001 hours.

Optimizer stripped from /mnt/mydata/huggingface/fastsam_runs/aerial_ft/weights/last.pt, 23.8MB

Optimizer stripped from /mnt/mydata/huggingface/fastsam_runs/aerial_ft/weights/best.pt, 23.8MB

Validating /mnt/mydata/huggingface/fastsam_runs/aerial_ft/weights/best.pt...

Ultralytics 8.4.115 🚀 Python-3.10.12 torch-2.3.0+cu121 CUDA:0 (NVIDIA RTX 6000 Ada Generation, 48639MiB)

YOLOv8s-seg summary (fused): 86 layers, 11,779,987 parameters, 0 gradients, 39.9 GFLOPs

                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95)     Mask(P          R      mAP50  mAP50-95): 100% ━━━━━━━━━━━━ 2/2 26.6it/s 0.1s

                   all         14        106      0.235      0.368      0.166      0.102      0.193      0.292      0.101     0.0484

Speed: 0.1ms preprocess, 1.3ms inference, 0.0ms loss, 1.1ms postprocess per image

training complete; results dir: /mnt/mydata/huggingface/fastsam_runs/aerial_ft

Step 4 - Inference with the fine-tuned model

We run the fine-tuned checkpoint (best.pt) on a held-out val tile and overlay the predicted instance masks - now specialized to the aerial domain.

saved 05_finetuned_inference.png; best = /mnt/mydata/huggingface/fastsam_runs/aerial_ft/weights/best.pt

Notes & next steps

This is a short demonstration fine-tune. For a real run: raise FASTSAM_FT_EPOCHS (e.g. 50-100), FASTSAM_FT_IMGSZ (e.g. 1024), and enable augmentation/plots.
The mask->polygon conversion treats each connected foreground component as one instance and a single object class. If the aerial masks encode multiple semantic classes (distinct pixel values), map each value to its own class id and expand names in data.yaml.
Because FastSAM is class-agnostic by design, fine-tuning adapts the region proposals to the aerial domain (rooftops/roads/vehicles), improving 'everything' segmentation on overhead imagery.

FastSAM — SAM's speed problem, solved

The YOLOv8-seg pipeline

Training losses

Prompting after the fact

Under the hood: it's a detector

Full notebook deep-dive

FastSAM 01 - Overview & Architecture

FastSAM vs SAM at a glance

The YOLOv8-seg pipeline

Load the real FastSAM model and inspect it

The prototype-mask assembly, illustrated

FastSAM 02 - Architecture Deep-Dive: Tensor Sizes & Features

Preprocessing

Hook the real network and record shapes

The segment-head output shapes

Visualize backbone feature maps + prototypes on the real image

FastSAM 03 - Training Approach & Loss Functions

Inputs and targets

The four loss terms

Visualize the DFL idea

Training recipe (FastSAM defaults)

FastSAM 04 - Inference (Everything / Box / Point / Text)

Mode 1 - 'Everything' (all instances in one pass)

Mode 2 - Box prompt

Mode 3 - Point prompt

Mode 4 - Text prompt (CLIP-guided)

FastSAM 05 - Fine-Tuning on Aerial Imagery

Step 1 - Download the dataset from the HF Hub

Step 2 - Convert semantic masks to YOLO-seg polygon labels

Visualize a converted sample (image + polygons)

Step 3 - Write data.yaml and fine-tune

Step 4 - Inference with the fine-tuned model

Notes & next steps

Step 3 - Write `data.yaml` and fine-tune