Bootstrapping a segmentation dataset with SAM2

Hand-drawing masks is the slowest part of any segmentation project. The trick that saved us weeks: let SAM2 draft the masks, then have annotators correct instead of draw. A rough box or a couple of clicks per object, and you get a clean polygon to fix rather than trace from scratch.

Prompt with boxes you already have

If a lightweight detector already gives boxes, feed them to SAM2 as prompts and keep the highest-scoring mask per box:

from sam2.sam2_image_predictor import SAM2ImagePredictor

predictor = SAM2ImagePredictor.from_pretrained("facebook/sam2-hiera-large")
predictor.set_image(image_rgb)

masks, scores, _ = predictor.predict(
    box=boxes_xyxy,           # from the rough detector
    multimask_output=False,
)

Turn masks into CVAT polygons

Vectorise the binary mask and simplify it, otherwise you ship 2000-point polygons that are miserable to edit:

import cv2
cnts,_ = cv2.findContours(mask.astype("uint8"),
                          cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
poly = cv2.approxPolyDP(cnts[0], epsilon=2.0, closed=True).reshape(-1,2)

Import those as pre-annotations, and the annotator's job becomes review and nudge. Two things to watch: SAM2 loves to grab shadows and reflections as part of the object, and it merges touching instances. Both are fast for a human to split, but you have to actually look, so do not skip the review pass and trust the model blindly.