Ensembling YOLO, MaskDINO and SegFormer

Three models, three strengths. YOLO is fast and finds small stuff, MaskDINO gives clean instances, SegFormer nails the fuzzy area classes that have no clear boundary. Combining them beat any single model, but only after we stopped naively averaging.

Detections: weighted boxes fusion, not NMS

NMS across models throws away agreement. Weighted Boxes Fusion keeps it, merging overlapping boxes into one and boosting the score when models agree:

from ensemble_boxes import weighted_boxes_fusion

boxes, scores, labels = weighted_boxes_fusion(
    [yolo_boxes, dino_boxes],
    [yolo_scores, dino_scores],
    [yolo_labels, dino_labels],
    weights=[1, 2], iou_thr=0.55, skip_box_thr=0.1)

Masks and semantic classes stay separate

Do not fuse a SegFormer semantic map with instance masks pixel-by-pixel. Instead use the semantic map as a gate: drop instance detections whose pixels mostly fall on a class the semantic model is confident is background. It removes a whole category of false positives on textured surfaces.

Calibrate before you weight

The models output scores on different scales, so a raw 0.6 from one is not a 0.6 from another. Fit a simple per-model temperature on the validation set first, otherwise your fusion weights are really just papering over miscalibration. Tune iou_thr and the weights against the actual metric you report, not against a gut feeling from looking at a dozen images.