Skip to content

feat(reid): add Deep OC-SORT adaptive appearance fusion to BoT-SORT - #571

Open
AlexBodner wants to merge 2 commits into
feat/core/reid-consume-reid-packagefrom
feat/core/deep-ocsort-adaptive-fusion
Open

feat(reid): add Deep OC-SORT adaptive appearance fusion to BoT-SORT#571
AlexBodner wants to merge 2 commits into
feat/core/reid-consume-reid-packagefrom
feat/core/deep-ocsort-adaptive-fusion

Conversation

@AlexBodner

@AlexBodner AlexBodner commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #511. Adds reid_fusion="adaptive" to BoTSORTTracker: the Adaptive Weighting rule from Deep OC-SORT (arXiv:2302.11813, eqs. 4–6), selectable alongside BoT-SORT's own min(d_iou, d_app) fusion. Default stays "botsort"; behaviour is unchanged unless opted into.

What

  • fuse_adaptive_reid_association in trackers/core/reid/fusion.py: IoU + (a_w + w_b) · A_c, where w_b is the capped gap between the best and second-best appearance match. Implements the difference form the authors ship as compute_aw_new_metric (the variant behind their published results), not the ratio form other ports carry. Only the weighting is ported — not Dynamic Appearance, the velocity term, or OCR/OOS — so it is usable by any tracker and named for what it does.
  • Two new BoTSORTTracker parameters, reid_appearance_weight (a_w, default 0.75) and reid_adaptive_weight_cap (epsilon, default 0.5), the paper's MOT17/MOT20 values. reid_appearance_threshold is ignored under adaptive fusion and the docstring says so.
  • Two documented departures from the reference: negative cosine similarities are clamped to zero, and appearance is gated by reid_proximity_threshold (the reference drops implausible pairs after matching; this integration thresholds the fused similarity directly). The discriminativeness gap is measured over all candidates before that gate, so a geometrically excluded candidate cannot read as similarity 0 and earn the bonus for being spatially alone.
  • docs/guides/reid.md: fusion choice, a measured comparison, and a new section on reid_proximity_threshold.

Measured

Three arms per dataset sharing detections, encoder, CMC and geometry; only appearance handling differs. Parameter columns give the full ReID settings each cell ran with, defaults included. SoccerNet and MOT17 rows use the published tuned BoT-SORT geometry from docs/evaluations/results.md (SoccerNet's reproduces the published 85.0 exactly); DanceTrack's tuned set equals the defaults. HOTA, best configuration per arm:

Dataset no ReID botsort adaptive botsort parameters adaptive parameters
SoccerNet test (49, oracle dets, in-domain OSNet) 85.00 87.29 ¹ 86.23 reid_appearance_threshold=0.075, reid_proximity_threshold=1.0 reid_appearance_weight=0.75, reid_adaptive_weight_cap=0.5, reid_proximity_threshold=1.0
DanceTrack val (25, YOLOX, MSMT17 OSNet) 53.89 56.11 57.39 reid_appearance_threshold=0.25, reid_proximity_threshold=0.5 reid_appearance_weight=2.4, reid_adaptive_weight_cap=0, reid_proximity_threshold=0.5
MOT17 val-half (7, YOLOX, in-domain OSNet) 69.05 69.00 68.72 reid_appearance_threshold=0.25, reid_proximity_threshold=0.5 reid_appearance_weight=0.75, reid_adaptive_weight_cap=0.5, reid_proximity_threshold=0.5

¹ botsort at reid_appearance_threshold=0.075, reid_proximity_threshold=1.0. It has the higher HOTA and IDF1 (83.13 vs 82.52) but 4 564 ID switches against adaptive's 1 692 and geometry's 2 523: min(d_iou, d_app) takes the cheapest candidate each frame and flips between same-kit players (two thirds of its identity changes revert within ten frames on SNMOT-120). Adaptive commits and holds. Neither dominates; adaptive is the rule for stable ids, which is the usual reason to enable appearance.

Adaptive cuts ID switches on SoccerNet (2523 → 1692, −33%) and DanceTrack (−12%); botsort fusion raises them on all three. Most of the SoccerNet HOTA gain at gate 1.0 is DetA: a returning player is re-attached immediately instead of spending frames as an unconfirmed new track (FN 13 026 → 623 for adaptive). On SoccerNet the decisive parameter is reid_proximity_threshold. The gate drops a pair when 1 − IoU exceeds it, so 0.99 still requires IoU ≥ 0.01 and only 1.0 disables it. A player who walks out of frame has a prediction extrapolated out of frame and zero overlap with the returning detection; the reference Deep OC-SORT recovers that with a geometry-only OCR pass against the last observed box, which BoT-SORT does not have, so here appearance at 1.0 is the only route. At 0.99 adaptive is +0.87 HOTA over tuned geometry (in-frame occlusions only); at 1.0 it is +1.23 with IDF1 +2.8, and botsort +2.29 with the switch count above. Both carry their closed-gate parameter sweeps to 1.0 unchanged. The same opening hurts DanceTrack and MOT17, where targets stay in frame: at 1.0 with their best DanceTrack settings adaptive loses 0.83 HOTA and min fusion loses 10 with seven times the ID switches. So it is documented as domain guidance, not a new default, with adaptive as the rule to prefer when the gate is open. Disabling the adaptive bonus (cap=0) changes HOTA by +0.12 / −0.13, so the gain is the additive form rather than the discriminativeness weighting. MOT17 is within single-sequence noise either way.

No post-processing interpolation; absolute numbers are not comparable to the paper's.

Tests

tests/core/test_botsort_reid.py: adaptive fusion unit tests including the gate-before-gap regression (a gated competitor must not inflate the bonus), top-two gap with fewer than two candidates, proximity uses standard IoU under GIoU/DIoU/CIoU, negative-weight rejection, and end-to-end tracker tests for both fusions. Full suite: 1531 passed.

🤖 Generated with Claude Code

@AlexBodner
AlexBodner requested a review from SkalskiP as a code owner August 19, 2026 14:45
@AlexBodner
AlexBodner force-pushed the feat/core/deep-ocsort-adaptive-fusion branch 3 times, most recently from d9470e7 to d8bb184 Compare August 19, 2026 16:10
BoT-SORT fuses appearance by taking the minimum of the geometry and
appearance costs, gated by two fixed thresholds. Deep OC-SORT instead adds
a weighted appearance term whose weight grows when the best match stands
clear of the runner-up, which helps most in crowded scenes where a fixed
weight is either too timid or too aggressive.

Adds fuse_adaptive_reid_association alongside the existing fusion and a
reid_fusion selector on BoTSORTTracker, with reid_appearance_weight and
reid_adaptive_weight_cap for its two knobs. Default stays "botsort", so
behaviour is unchanged unless opted into.

Implements equations (4)-(6) of the paper, matching compute_aw_new_metric
in the authors' integrated_ocsort_embedding tracker (the variant behind
their published results) rather than the ratio-based variant that other
ports use.

Named for what it does rather than where it came from: only the adaptive
weighting is ported, onto a BoT-SORT base, without Dynamic Appearance, the
velocity term or OCR/OOS.

Two documented departures from the reference: negative cosine similarities
are clamped to zero, and appearance is gated by reid_proximity_threshold.
Deep OC-SORT drops implausible pairs after matching, but this integration
thresholds the fused similarity directly, so the gate has to happen during
fusion. The discriminativeness gap is measured across all candidates
before that gate is applied, so a candidate excluded on geometry cannot
read as similarity 0 and earn the bonus for being spatially alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@AlexBodner
AlexBodner force-pushed the feat/core/deep-ocsort-adaptive-fusion branch 4 times, most recently from 28cf780 to 17a80a0 Compare August 19, 2026 18:37
Documents reid_fusion and its parameters, and compares both fusion rules
against a geometry-only baseline on SoccerNet, DanceTrack and MOT17 on the
published tuned BoT-SORT geometry. The geometry-only column is the one that
decides whether to enable appearance at all, so it belongs in the table.

Adds a section on reid_proximity_threshold. Its 0.5 default restricts
appearance to pairs already overlapping at IoU >= 0.5, and 0.99 still
requires IoU >= 0.01, which a target returning from outside the frame does
not meet: its prediction has been extrapolated out of frame and overlaps the
returning detection by exactly zero. Only 1.0 disables the gate. On SoccerNet
test that is worth +1.23 HOTA and a third fewer ID switches on the published
tuned geometry, and it is what lets a player who walked out of frame come
back with the same id. The reference Deep OC-SORT recovers that case with a
geometry-only pass against the last observed box; BoT-SORT has no such
stage, so appearance at an open gate is the only route.

The same change hurts DanceTrack, whose targets stay in frame, so this is
documented as domain guidance rather than a new default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@AlexBodner
AlexBodner force-pushed the feat/core/deep-ocsort-adaptive-fusion branch from 17a80a0 to d7387c8 Compare August 19, 2026 18:53
@Borda Borda added the enhancement New feature or request label Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants