feat(reid): add Deep OC-SORT adaptive appearance fusion to BoT-SORT - #571
Open
AlexBodner wants to merge 2 commits into
Open
feat(reid): add Deep OC-SORT adaptive appearance fusion to BoT-SORT#571AlexBodner wants to merge 2 commits into
AlexBodner wants to merge 2 commits into
Conversation
AlexBodner
force-pushed
the
feat/core/deep-ocsort-adaptive-fusion
branch
3 times, most recently
from
August 19, 2026 16:10
d9470e7 to
d8bb184
Compare
BoT-SORT fuses appearance by taking the minimum of the geometry and appearance costs, gated by two fixed thresholds. Deep OC-SORT instead adds a weighted appearance term whose weight grows when the best match stands clear of the runner-up, which helps most in crowded scenes where a fixed weight is either too timid or too aggressive. Adds fuse_adaptive_reid_association alongside the existing fusion and a reid_fusion selector on BoTSORTTracker, with reid_appearance_weight and reid_adaptive_weight_cap for its two knobs. Default stays "botsort", so behaviour is unchanged unless opted into. Implements equations (4)-(6) of the paper, matching compute_aw_new_metric in the authors' integrated_ocsort_embedding tracker (the variant behind their published results) rather than the ratio-based variant that other ports use. Named for what it does rather than where it came from: only the adaptive weighting is ported, onto a BoT-SORT base, without Dynamic Appearance, the velocity term or OCR/OOS. Two documented departures from the reference: negative cosine similarities are clamped to zero, and appearance is gated by reid_proximity_threshold. Deep OC-SORT drops implausible pairs after matching, but this integration thresholds the fused similarity directly, so the gate has to happen during fusion. The discriminativeness gap is measured across all candidates before that gate is applied, so a candidate excluded on geometry cannot read as similarity 0 and earn the bonus for being spatially alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AlexBodner
force-pushed
the
feat/core/deep-ocsort-adaptive-fusion
branch
4 times, most recently
from
August 19, 2026 18:37
28cf780 to
17a80a0
Compare
Documents reid_fusion and its parameters, and compares both fusion rules against a geometry-only baseline on SoccerNet, DanceTrack and MOT17 on the published tuned BoT-SORT geometry. The geometry-only column is the one that decides whether to enable appearance at all, so it belongs in the table. Adds a section on reid_proximity_threshold. Its 0.5 default restricts appearance to pairs already overlapping at IoU >= 0.5, and 0.99 still requires IoU >= 0.01, which a target returning from outside the frame does not meet: its prediction has been extrapolated out of frame and overlaps the returning detection by exactly zero. Only 1.0 disables the gate. On SoccerNet test that is worth +1.23 HOTA and a third fewer ID switches on the published tuned geometry, and it is what lets a player who walked out of frame come back with the same id. The reference Deep OC-SORT recovers that case with a geometry-only pass against the last observed box; BoT-SORT has no such stage, so appearance at an open gate is the only route. The same change hurts DanceTrack, whose targets stay in frame, so this is documented as domain guidance rather than a new default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AlexBodner
force-pushed
the
feat/core/deep-ocsort-adaptive-fusion
branch
from
August 19, 2026 18:53
17a80a0 to
d7387c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #511. Adds
reid_fusion="adaptive"toBoTSORTTracker: the Adaptive Weighting rule from Deep OC-SORT (arXiv:2302.11813, eqs. 4–6), selectable alongside BoT-SORT's ownmin(d_iou, d_app)fusion. Default stays"botsort"; behaviour is unchanged unless opted into.What
fuse_adaptive_reid_associationintrackers/core/reid/fusion.py:IoU + (a_w + w_b) · A_c, wherew_bis the capped gap between the best and second-best appearance match. Implements the difference form the authors ship ascompute_aw_new_metric(the variant behind their published results), not the ratio form other ports carry. Only the weighting is ported — not Dynamic Appearance, the velocity term, or OCR/OOS — so it is usable by any tracker and named for what it does.BoTSORTTrackerparameters,reid_appearance_weight(a_w, default 0.75) andreid_adaptive_weight_cap(epsilon, default 0.5), the paper's MOT17/MOT20 values.reid_appearance_thresholdis ignored under adaptive fusion and the docstring says so.reid_proximity_threshold(the reference drops implausible pairs after matching; this integration thresholds the fused similarity directly). The discriminativeness gap is measured over all candidates before that gate, so a geometrically excluded candidate cannot read as similarity 0 and earn the bonus for being spatially alone.docs/guides/reid.md: fusion choice, a measured comparison, and a new section onreid_proximity_threshold.Measured
Three arms per dataset sharing detections, encoder, CMC and geometry; only appearance handling differs. Parameter columns give the full ReID settings each cell ran with, defaults included. SoccerNet and MOT17 rows use the published tuned BoT-SORT geometry from
docs/evaluations/results.md(SoccerNet's reproduces the published 85.0 exactly); DanceTrack's tuned set equals the defaults. HOTA, best configuration per arm:botsortadaptivebotsortparametersadaptiveparametersreid_appearance_threshold=0.075,reid_proximity_threshold=1.0reid_appearance_weight=0.75,reid_adaptive_weight_cap=0.5,reid_proximity_threshold=1.0reid_appearance_threshold=0.25,reid_proximity_threshold=0.5reid_appearance_weight=2.4,reid_adaptive_weight_cap=0,reid_proximity_threshold=0.5reid_appearance_threshold=0.25,reid_proximity_threshold=0.5reid_appearance_weight=0.75,reid_adaptive_weight_cap=0.5,reid_proximity_threshold=0.5¹
botsortatreid_appearance_threshold=0.075,reid_proximity_threshold=1.0. It has the higher HOTA and IDF1 (83.13 vs 82.52) but 4 564 ID switches against adaptive's 1 692 and geometry's 2 523:min(d_iou, d_app)takes the cheapest candidate each frame and flips between same-kit players (two thirds of its identity changes revert within ten frames on SNMOT-120). Adaptive commits and holds. Neither dominates; adaptive is the rule for stable ids, which is the usual reason to enable appearance.Adaptive cuts ID switches on SoccerNet (2523 → 1692, −33%) and DanceTrack (−12%);
botsortfusion raises them on all three. Most of the SoccerNet HOTA gain at gate 1.0 is DetA: a returning player is re-attached immediately instead of spending frames as an unconfirmed new track (FN 13 026 → 623 for adaptive). On SoccerNet the decisive parameter isreid_proximity_threshold. The gate drops a pair when1 − IoUexceeds it, so 0.99 still requiresIoU ≥ 0.01and only 1.0 disables it. A player who walks out of frame has a prediction extrapolated out of frame and zero overlap with the returning detection; the reference Deep OC-SORT recovers that with a geometry-only OCR pass against the last observed box, which BoT-SORT does not have, so here appearance at 1.0 is the only route. At 0.99 adaptive is +0.87 HOTA over tuned geometry (in-frame occlusions only); at 1.0 it is +1.23 with IDF1 +2.8, andbotsort+2.29 with the switch count above. Both carry their closed-gate parameter sweeps to 1.0 unchanged. The same opening hurts DanceTrack and MOT17, where targets stay in frame: at 1.0 with their best DanceTrack settings adaptive loses 0.83 HOTA andminfusion loses 10 with seven times the ID switches. So it is documented as domain guidance, not a new default, with adaptive as the rule to prefer when the gate is open. Disabling the adaptive bonus (cap=0) changes HOTA by +0.12 / −0.13, so the gain is the additive form rather than the discriminativeness weighting. MOT17 is within single-sequence noise either way.No post-processing interpolation; absolute numbers are not comparable to the paper's.
Tests
tests/core/test_botsort_reid.py: adaptive fusion unit tests including the gate-before-gap regression (a gated competitor must not inflate the bonus), top-two gap with fewer than two candidates, proximity uses standard IoU under GIoU/DIoU/CIoU, negative-weight rejection, and end-to-end tracker tests for both fusions. Full suite: 1531 passed.🤖 Generated with Claude Code