diff --git a/README.md b/README.md index bcbcb324..c263a10e 100644 --- a/README.md +++ b/README.md @@ -132,12 +132,13 @@ This repository offers a growing collection of computer vision tutorials. Learn | [Image Classification with OpenAI Clip](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-use-openai-clip-classification.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-use-openai-clip-classification.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-use-openai-clip-classification.ipynb) | [![Roboflow](https://raw.githubusercontent.com/roboflow-ai/notebooks/main/assets/badges/roboflow-blogpost.svg)](https://blog.roboflow.com/how-to-use-openai-clip) [![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://www.youtube.com/watch?v=8o701AEoZ8I) | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/openai/CLIP) [![arXiv](https://img.shields.io/badge/arXiv-2103.00020-b31b1b.svg)](https://arxiv.org/abs/2103.00020)| | [Fine-Tune YOLOv4-tiny Darknet on Object Detection Dataset](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/train-yolov4-tiny-object-detection-on-custom-data.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/train-yolov4-tiny-object-detection-on-custom-data.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/train-yolov4-tiny-object-detection-on-custom-data.ipynb) | [![Roboflow](https://raw.githubusercontent.com/roboflow-ai/notebooks/main/assets/badges/roboflow-blogpost.svg)](https://blog.roboflow.ai/train-yolov4-tiny-on-custom-data-lighting-fast-detection) [![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://www.youtube.com/watch?v=NTnZgLsk_DA) | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/roboflow-ai/darknet) [![arXiv](https://img.shields.io/badge/arXiv-2011.04244-b31b1b.svg)](https://arxiv.org/abs/2011.04244)| | [Train a YOLOv8 Classification Model with No Labeling](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-train-yolov8-classification-no-labeling.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-train-yolov8-classification-no-labeling.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-train-yolov8-classification-no-labeling.ipynb) | [![Roboflow](https://raw.githubusercontent.com/roboflow-ai/notebooks/main/assets/badges/roboflow-blogpost.svg)](https://blog.roboflow.com/train-classification-model-no-labeling/) | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/ultralytics/ultralytics) | -## 📍 tracker tutorials (3 notebooks) +## 📍 tracker tutorials (4 notebooks) | **notebook** | **open in colab / kaggle / sagemaker studio lab** | **complementary materials** | **repository / paper** | |:------------:|:-------------------------------------------------:|:---------------------------:|:----------------------:| | [How to Track Objects with RF-DETR and ByteTrack Tracker](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-bytetrack-tracker.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-bytetrack-tracker.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-bytetrack-tracker.ipynb) | [![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://www.youtube.com/watch?v=u0k2dTZ0vfs) | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/roboflow/trackers) [![arXiv](https://img.shields.io/badge/arXiv-2110.06864-b31b1b.svg)](https://arxiv.org/abs/2110.06864)| | [How to Track Objects with RF-DETR and SORT Tracker](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-sort-tracker.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-sort-tracker.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-sort-tracker.ipynb) | [![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://www.youtube.com/watch?v=u0k2dTZ0vfs) | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/roboflow/trackers) [![arXiv](https://img.shields.io/badge/arXiv-1602.00763-b31b1b.svg)](https://arxiv.org/abs/1602.00763)| | [How to Track Objects with RF-DETR and OC-SORT Tracker](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-ocsort-tracker.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-ocsort-tracker.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-track-objects-with-ocsort-tracker.ipynb) | [![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://www.youtube.com/watch?v=u0k2dTZ0vfs) | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/roboflow/trackers) [![arXiv](https://img.shields.io/badge/arXiv-2206.14360-b31b1b.svg)](https://arxiv.org/abs/2206.14360)| +| [How to Add ReID to Trackers](https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-add-reid-to-trackers.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-add-reid-to-trackers.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/roboflow-ai/notebooks/blob/main/notebooks/how-to-add-reid-to-trackers.ipynb) | | [![GitHub](https://badges.aleen42.com/src/github.svg)](https://github.com/roboflow/trackers) [![arXiv](https://img.shields.io/badge/arXiv-2206.14651-b31b1b.svg)](https://arxiv.org/abs/2206.14651)| ## 🛠️ computer vision skills (23 notebooks) | **notebook** | **open in colab / kaggle / sagemaker studio lab** | **complementary materials** | **repository / paper** | |:------------:|:-------------------------------------------------:|:---------------------------:|:----------------------:| diff --git a/automation/notebooks-table-data.csv b/automation/notebooks-table-data.csv index 3aa2f710..35682eda 100644 --- a/automation/notebooks-table-data.csv +++ b/automation/notebooks-table-data.csv @@ -15,6 +15,7 @@ Zero-Shot Object Detection and Segmentation with Google Gemini 2.5, zero-shot-ob How to Track Objects with RF-DETR and ByteTrack Tracker, how-to-track-objects-with-bytetrack-tracker.ipynb, , https://www.youtube.com/watch?v=u0k2dTZ0vfs, https://github.com/roboflow/trackers, 2110.06864, False, True, trackers How to Track Objects with RF-DETR and SORT Tracker, how-to-track-objects-with-sort-tracker.ipynb, , https://www.youtube.com/watch?v=u0k2dTZ0vfs, https://github.com/roboflow/trackers, 1602.00763, False, True, trackers How to Track Objects with RF-DETR and OC-SORT Tracker, how-to-track-objects-with-ocsort-tracker.ipynb, , https://www.youtube.com/watch?v=u0k2dTZ0vfs, https://github.com/roboflow/trackers, 2206.14360, False, True, trackers +How to Add ReID to Trackers, how-to-add-reid-to-trackers.ipynb, , , https://github.com/roboflow/trackers, 2206.14651, False, True, trackers Fine-Tune RF-DETR on Object Detection Dataset, how-to-finetune-rf-detr-on-detection-dataset.ipynb, https://blog.roboflow.com/rf-detr, https://www.youtube.com/live/yHW0ip-2i54, https://github.com/roboflow/rf-detr, 2511.09554, False, True, models Zero-Shot Object Detection and Segmentation with YOLOE, zero-shot-object-detection-and-segmentation-with-yoloe.ipynb, https://blog.roboflow.com/yoloe-zero-shot-object-detection-segmentation/, https://www.youtube.com/watch?v=eHAnIehnCt4, https://github.com/THU-MIG/yoloe, 2503.07465, False, True, models Fine-Tune YOLOv12 on Object Detection Dataset, train-yolov12-object-detection-model.ipynb, https://blog.roboflow.com/train-yolov12-model, , https://github.com/sunsmarterjie/yolov12, 2502.12524, False, True, models diff --git a/notebooks/how-to-add-reid-to-trackers.ipynb b/notebooks/how-to-add-reid-to-trackers.ipynb new file mode 100644 index 00000000..80e30be0 --- /dev/null +++ b/notebooks/how-to-add-reid-to-trackers.ipynb @@ -0,0 +1,863 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[![Roboflow Notebooks](https://media.roboflow.com/notebooks/template/bannertest2-2.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672932710194)](https://github.com/roboflow/notebooks)\n", + "\n", + "# How to Add ReID to Trackers\n", + "\n", + "BoT-SORT can fuse visual ReID embeddings with IoU during association. In this\n", + "notebook you will enable appearance ReID with the [`reid`](https://github.com/roboflow/re-ID)\n", + "package, then run BoT-SORT + ReID on MOT17 val-half using YOLOX detections and\n", + "TrackEval metrics.\n", + "\n", + "For threshold selection and MOT17 / SoccerNet results, see the\n", + "[ReID appearance guide](https://trackers.roboflow.com/latest/learn/reid/)." + ], + "id": "hdr-banner" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup\n" + ], + "id": "setup-h" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Check GPU availability\n", + "\n", + "Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do\n", + "that. In case of any problems navigate to `Runtime` -> `Change runtime type` ->\n", + "`Hardware accelerator`, set it to `GPU`, and then click `Save`.\n" + ], + "id": "setup-gpu-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "!nvidia-smi" + ], + "execution_count": 12, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Thu Jul 30 16:37:50 2026 \n", + "+-----------------------------------------------------------------------------------------+\n", + "| NVIDIA-SMI 580.82.07 Driver Version: 580.82.07 CUDA Version: 13.0 |\n", + "+-----------------------------------------+------------------------+----------------------+\n", + "| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n", + "| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n", + "| | | MIG M. |\n", + "|=========================================+========================+======================|\n", + "| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n", + "| N/A 54C P0 27W / 70W | 2267MiB / 15360MiB | 0% Default |\n", + "| | | N/A |\n", + "+-----------------------------------------+------------------------+----------------------+\n", + "\n", + "+-----------------------------------------------------------------------------------------+\n", + "| Processes: |\n", + "| GPU GI CI PID Type Process name GPU Memory |\n", + "| ID ID Usage |\n", + "|=========================================================================================|\n", + "| 0 N/A N/A 1483 C /usr/bin/python3 1144MiB |\n", + "| 0 N/A N/A 24122 C /usr/bin/python3 1120MiB |\n", + "+-----------------------------------------------------------------------------------------+\n" + ] + } + ], + "id": "setup-gpu" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install dependencies\n", + "\n", + "Install Trackers with the ReID extra. Appearance ReID is not on PyPI trackers yet,\n", + "so this notebook pins the feature branch. After release, switch to\n", + "`trackers[reid]==2.6.0`.\n", + "\n", + "You may see dependency conflict warnings in Google Colab. This is expected for the\n", + "preinstalled Google Colab environment and does not affect functionality.\n" + ], + "id": "setup-install-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "!pip install -q matplotlib gdown\n", + "!pip install -q \"trackers[reid] @ git+https://github.com/roboflow/trackers.git@feat/core/reid-consume-reid-package\"\n" + ], + "execution_count": 13, + "outputs": [ + { + "output_type": "stream", + "text": [ + " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", + " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", + " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n" + ] + } + ], + "id": "setup-install" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Download the dataset\n", + "\n", + "Download MOT17 val ground truth and frames with the Trackers CLI, then fetch the\n", + "YOLOX val detections. YOLOX frame IDs\n", + "are remapped to `1...N` so they align with MOT frame indexing.\n" + ], + "id": "setup-download-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from pathlib import Path\n", + "\n", + "REPO_ROOT = Path(\"..\")\n", + "\n", + "VAL_SEQUENCES = [\n", + " \"MOT17-02-FRCNN\",\n", + " \"MOT17-04-FRCNN\",\n", + " \"MOT17-05-FRCNN\",\n", + " \"MOT17-09-FRCNN\",\n", + " \"MOT17-10-FRCNN\",\n", + " \"MOT17-11-FRCNN\",\n", + " \"MOT17-13-FRCNN\",\n", + "]\n", + "\n", + "MOT17_VAL = REPO_ROOT / \"mot17\" / \"val\"\n", + "YOLOX_DIR = REPO_ROOT / \"MOT17_yolox_dets\"\n", + "YOLOX_VAL_DIR = YOLOX_DIR / \"val\"\n", + "YOLOX_ZIP = YOLOX_DIR / \"yolox_detections_MOT17.zip\"\n", + "YOLOX_GDRIVE_ID = \"1BuXtPWf8QbPU_y1i2xY2IbTE-rj3l6qT\"\n", + "OUTPUT_ROOT = REPO_ROOT / \"trackers_reid_outputs\"\n", + "OUTPUT_ROOT.mkdir(parents=True, exist_ok=True)\n", + "\n", + "!trackers download mot17 --split val --asset annotations,frames -o {REPO_ROOT}\n", + "\n", + "!mkdir -p {YOLOX_DIR}\n", + "!gdown {YOLOX_GDRIVE_ID} -O {YOLOX_ZIP}\n", + "!unzip -qo {YOLOX_ZIP} -d {YOLOX_DIR}\n" + ], + "execution_count": 14, + "outputs": [ + { + "output_type": "stream", + "text": [ + "FutureWarning: target=None is deprecated since `v0.8`; use `TargetMode.NOTIFY` instead. Will be removed in `v1.0`.\n", + "FutureWarning: target=None is deprecated since `v0.8`; use `TargetMode.NOTIFY` instead. Will be removed in `v1.0`.\n", + "UserWarning: The trackers CLI is in beta. APIs may change in future releases.\n", + "[download] mot17:val:annotations\n", + " using cached mot17-val-annotations.zip\n", + "[extract] mot17:val:annotations\n", + "[done] mot17:val:annotations\n", + "[download] mot17:val:frames\n", + " using cached mot17-val-frames.zip\n", + "[extract] mot17:val:frames\n", + "[done] mot17:val:frames\n", + "Downloading...\n", + "From: https://drive.google.com/uc?id=1BuXtPWf8QbPU_y1i2xY2IbTE-rj3l6qT\n", + "To: /MOT17_yolox_dets/yolox_detections_MOT17.zip\n", + "100% 6.35M/6.35M [00:00<00:00, 203MB/s]\n" + ] + } + ], + "id": "setup-download" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def yolox_det_path(seq: str) -> Path:\n", + " return YOLOX_VAL_DIR / f\"{seq.replace('-FRCNN', '')}_val.txt\"\n", + "\n", + "\n", + "SEQUENCE_PATHS: dict[str, dict] = {}\n", + "for seq in VAL_SEQUENCES:\n", + " gt = MOT17_VAL / seq / \"gt\" / \"gt.txt\"\n", + " img = MOT17_VAL / seq / \"img1\"\n", + " det = yolox_det_path(seq)\n", + " if not (gt.is_file() and img.is_dir() and det.is_file()):\n", + " print(f\" skip {seq}: missing gt, img1, or YOLOX det\")\n", + " continue\n", + " n_frames = len(list(img.glob(\"*.jpg\")))\n", + " SEQUENCE_PATHS[seq] = {\"gt\": gt, \"img\": img, \"det\": det, \"n_frames\": n_frames}\n", + " print(f\" {seq}: {n_frames} frames\")\n", + "\n", + "ACTIVE_SEQUENCES = list(SEQUENCE_PATHS)\n", + "if not ACTIVE_SEQUENCES:\n", + " raise RuntimeError(\"No sequences ready - re-run the download cell above.\")\n", + "\n", + "SEQMAP_PATH = OUTPUT_ROOT / \"MOT17-val.txt\"\n", + "SEQMAP_PATH.write_text(\"name\\n\" + \"\\n\".join(ACTIVE_SEQUENCES) + \"\\n\")\n", + "print(f\"\\n{len(ACTIVE_SEQUENCES)} sequences ready\")\n" + ], + "execution_count": 15, + "outputs": [ + { + "output_type": "stream", + "text": [ + " MOT17-02-FRCNN: 299 frames\n", + " MOT17-04-FRCNN: 524 frames\n", + " MOT17-05-FRCNN: 418 frames\n", + " MOT17-09-FRCNN: 262 frames\n", + " MOT17-10-FRCNN: 326 frames\n", + " MOT17-11-FRCNN: 449 frames\n", + " MOT17-13-FRCNN: 374 frames\n", + "\n", + "7 sequences ready\n" + ] + } + ], + "id": "setup-index" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load ReID model\n", + "\n", + "Load a MOT17 FastReID SBS50 encoder from the `reid` package. For other checkpoints\n", + "(for example OSNet on MSMT17), pass a different model id to\n", + "`ReIDModel.from_pretrained(...)`. See the\n", + "[`reid` training guide](https://github.com/roboflow/re-ID/blob/main/docs/learn/train.md)." + ], + "id": "reid-model-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "import warnings\n", + "\n", + "import cv2\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import supervision as sv\n", + "import torch\n", + "from reid import FASTREID_MOT17_SBS50, ReIDModel\n", + "\n", + "from trackers import BoTSORTTracker\n", + "from trackers.eval import evaluate_mot_sequences\n", + "from trackers.io.frames import load_mot_frame_image\n", + "from trackers.io.mot import _MOTOutput, load_mot_file\n", + "\n", + "warnings.filterwarnings(\"ignore\")\n", + "\n", + "device = torch.cuda.get_device_name(0) if torch.cuda.is_available() else \"cpu\"\n", + "print(f\"PyTorch {torch.__version__} | CUDA {torch.cuda.is_available()} | {device}\")\n", + "\n", + "# Use another reid model id here to try OSNet or a custom checkpoint.\n", + "REID_ENCODER = FASTREID_MOT17_SBS50\n", + "REID_APPEARANCE_THRESHOLD = 0.2\n", + "reid_model = ReIDModel.from_pretrained(REID_ENCODER)\n", + "\n", + "print(f\"Encoder: {REID_ENCODER} | appearance_threshold: {REID_APPEARANCE_THRESHOLD}\")\n", + "print(reid_model.preprocessing.describe())\n" + ], + "execution_count": 16, + "outputs": [ + { + "output_type": "stream", + "text": [ + "PyTorch 2.11.0+cu128 | CUDA True | Tesla T4\n", + "Encoder: fastreid_mot17_sbs50 | appearance_threshold: 0.2\n", + "ReIDPreprocessing(resize=384x128 [stretch, bilinear], BGR→RGB, mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n" + ] + } + ], + "id": "reid-model" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run BoT-SORT with ReID\n", + "\n", + "Create a `BoTSORTTracker` with a `reid_model` and pass the current frame to\n", + "`update()` so embeddings can be extracted.\n" + ], + "id": "helpers-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def load_yolox_dets(det_path: Path) -> dict[int, sv.Detections]:\n", + " \"\"\"Load YOLOX dets (`frame,x1,y1,x2,y2,score`) as 1-based frame -> Detections.\"\"\"\n", + " rows: list[tuple[int, float, float, float, float, float]] = []\n", + " with det_path.open() as f:\n", + " for line in f:\n", + " parts = line.strip().split(\",\")\n", + " if len(parts) < 6:\n", + " continue\n", + " frame, x1, y1, x2, y2, score = map(float, parts[:6])\n", + " if score > 0:\n", + " rows.append((int(frame), x1, y1, x2, y2, score))\n", + " if not rows:\n", + " return {}\n", + " min_frame = min(frame for frame, *_ in rows)\n", + " offset = min_frame - 1 if min_frame > 1 else 0\n", + " by_frame: dict[int, list[list[float]]] = {}\n", + " for frame, x1, y1, x2, y2, score in rows:\n", + " by_frame.setdefault(frame - offset, []).append([x1, y1, x2, y2, score])\n", + " return {\n", + " frame: sv.Detections(\n", + " xyxy=np.asarray(boxes, dtype=np.float32)[:, :4],\n", + " confidence=np.asarray(boxes, dtype=np.float32)[:, 4],\n", + " )\n", + " for frame, boxes in by_frame.items()\n", + " }\n", + "\n", + "\n", + "pred_dir = OUTPUT_ROOT / \"botsort_reid\" / \"preds\"\n", + "pred_dir.mkdir(parents=True, exist_ok=True)\n", + "\n", + "for seq in ACTIVE_SEQUENCES:\n", + " spec = SEQUENCE_PATHS[seq]\n", + " dets = load_yolox_dets(spec[\"det\"])\n", + " images = sorted(spec[\"img\"].glob(\"*.jpg\"))\n", + " tracker = BoTSORTTracker(\n", + " enable_cmc=True,\n", + " reid_model=reid_model,\n", + " reid_ema_alpha=0.9,\n", + " appearance_threshold=REID_APPEARANCE_THRESHOLD,\n", + " )\n", + " with _MOTOutput(pred_dir / f\"{seq}.txt\") as out:\n", + " for frame_idx in range(1, spec[\"n_frames\"] + 1):\n", + " frame = cv2.imread(str(images[frame_idx - 1]))\n", + " tracked = tracker.update(dets.get(frame_idx, sv.Detections.empty()), frame)\n", + " if tracked.tracker_id is not None:\n", + " tracked = tracked[tracked.tracker_id != -1]\n", + " out.write(frame_idx, tracked)\n", + " print(f\" {seq}: {spec['n_frames']} frames\")\n", + "\n", + "result_reid = evaluate_mot_sequences(\n", + " gt_dir=MOT17_VAL,\n", + " tracker_dir=pred_dir,\n", + " seqmap=SEQMAP_PATH,\n", + " metrics=[\"CLEAR\", \"HOTA\", \"Identity\"],\n", + ")\n", + "agg = result_reid.aggregate\n", + "print(\n", + " f\"BoT-SORT + ReID: HOTA {agg.HOTA.HOTA * 100:6.2f} \"\n", + " f\"MOTA {agg.CLEAR.MOTA * 100:6.2f} \"\n", + " f\"IDF1 {agg.Identity.IDF1 * 100:6.2f} \"\n", + " f\"IDSW {agg.CLEAR.IDSW}\"\n", + ")\n" + ], + "execution_count": null, + "outputs": [], + "id": "f38487ea" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Appearance distance histogram\n", + "\n", + "Encoder diagnostic on MOT17 val **GT crops** (no detector). Pair sampling matches\n", + "what BoT-SORT association can see: one video at a time, within the lost-track\n", + "horizon.\n", + "\n", + "- Axis: `d_app = 0.5 * (1 - cos)` (BoT-SORT `embedding_distance / 2`)\n", + "- Positives: same ID, same sequence, `1 <= |frame gap| <= MAX_FRAME_GAP`\n", + "- Negatives: different ID, same sequence, `1 <= |frame gap| <= MAX_FRAME_GAP`\n", + "- Sampling: equal quota per sequence, and same-ID pairs pick an identity uniformly\n", + " so that long tracks and crowded sequences do not dominate either curve\n", + "- θ lines: 0.25 (default) and `REID_APPEARANCE_THRESHOLD`\n", + "\n", + "`MAX_FRAME_GAP` defaults to 30 (BoT-SORT `lost_track_buffer` at 30 FPS).\n", + "" + ], + "id": "hist-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from collections import defaultdict\n", + "\n", + "N_INTRA, N_INTER = 5000, 10000\n", + "MIN_FRAME_GAP, MAX_FRAME_GAP = 1, 30 # ~lost_track_buffer @ 30 FPS\n", + "CANDIDATE_THETAS = (0.20, 0.25)\n", + "DAPP_BINS = np.linspace(0.0, 1.0, 51)\n", + "\n", + "\n", + "def pedestrian_detections(gt_frame) -> sv.Detections:\n", + " keep = (gt_frame.confidences > 0) & (gt_frame.classes == 1)\n", + " if not np.any(keep):\n", + " return sv.Detections.empty()\n", + " return sv.Detections(\n", + " xyxy=sv.xywh_to_xyxy(gt_frame.boxes[keep]).astype(np.float32),\n", + " tracker_id=gt_frame.ids[keep].astype(int),\n", + " )\n", + "\n", + "\n", + "def collect_gt_embeddings(model: ReIDModel, sequences: list[str], frame_stride: int = 1):\n", + " embeddings, labels, frame_ids, seq_ids = [], [], [], []\n", + " label_by_key: dict[str, int] = {}\n", + "\n", + " for sid, seq in enumerate(sequences):\n", + " spec = SEQUENCE_PATHS[seq]\n", + " gt_by_frame = load_mot_file(spec[\"gt\"])\n", + " images = sorted(spec[\"img\"].glob(\"*.jpg\"))\n", + " for frame_idx in range(1, spec[\"n_frames\"] + 1, frame_stride):\n", + " gt = gt_by_frame.get(frame_idx)\n", + " if gt is None:\n", + " continue\n", + " dets = pedestrian_detections(gt)\n", + " if len(dets) == 0:\n", + " continue\n", + " bgr = cv2.imread(str(images[frame_idx - 1]))\n", + " if bgr is None:\n", + " continue\n", + " feats = model.extract_features(dets, bgr)\n", + " for i, tid in enumerate(dets.tracker_id):\n", + " key = f\"{seq}_{int(tid)}\"\n", + " label_by_key.setdefault(key, len(label_by_key))\n", + " embeddings.append(feats[i])\n", + " labels.append(label_by_key[key])\n", + " frame_ids.append(frame_idx)\n", + " seq_ids.append(sid)\n", + "\n", + " if not embeddings:\n", + " raise RuntimeError(\"No GT embeddings (conf>0, class==1 pedestrians).\")\n", + " return (\n", + " np.stack(embeddings),\n", + " np.asarray(labels, dtype=np.int64),\n", + " np.asarray(frame_ids, dtype=np.int64),\n", + " np.asarray(seq_ids, dtype=np.int64),\n", + " )\n", + "\n", + "\n", + "def _pair_distances(normed: np.ndarray, pairs: np.ndarray) -> np.ndarray:\n", + " return 0.5 * (1.0 - np.einsum(\"ij,ij->i\", normed[pairs[:, 0]], normed[pairs[:, 1]]))\n", + "\n", + "\n", + "def _draw_pairs_in_band(\n", + " rng: np.random.Generator,\n", + " *,\n", + " frames: np.ndarray,\n", + " ids: np.ndarray,\n", + " tracks: list[np.ndarray],\n", + " n_wanted: int,\n", + " same_id: bool,\n", + " min_frame_gap: int,\n", + " max_frame_gap: int,\n", + ") -> list[tuple[int, int]]:\n", + " \"\"\"Draw slot pairs from one sequence whose frame gap falls inside the band.\"\"\"\n", + "\n", + " def partner_of(candidates: np.ndarray, anchor_frame: int) -> int | None:\n", + " before_lo = int(np.searchsorted(candidates, anchor_frame - max_frame_gap, \"left\"))\n", + " before_hi = int(np.searchsorted(candidates, anchor_frame - min_frame_gap, \"right\"))\n", + " after_lo = int(np.searchsorted(candidates, anchor_frame + min_frame_gap, \"left\"))\n", + " after_hi = int(np.searchsorted(candidates, anchor_frame + max_frame_gap, \"right\"))\n", + " n_before, n_after = max(0, before_hi - before_lo), max(0, after_hi - after_lo)\n", + " if n_before + n_after == 0:\n", + " return None\n", + " draw = int(rng.integers(n_before + n_after))\n", + " return before_lo + draw if draw < n_before else after_lo + (draw - n_before)\n", + "\n", + " pairs: list[tuple[int, int]] = []\n", + " for _ in range(n_wanted * 64):\n", + " if len(pairs) >= n_wanted:\n", + " break\n", + " if same_id:\n", + " if not tracks:\n", + " break\n", + " track = tracks[int(rng.integers(len(tracks)))]\n", + " anchor = int(track[int(rng.integers(len(track)))])\n", + " slot = partner_of(frames[track], int(frames[anchor]))\n", + " partner = None if slot is None else int(track[slot])\n", + " else:\n", + " anchor = int(rng.integers(len(frames)))\n", + " partner = partner_of(frames, int(frames[anchor]))\n", + " if partner is not None and ids[partner] == ids[anchor]:\n", + " partner = None\n", + " if partner is None or partner == anchor:\n", + " continue\n", + " pairs.append((anchor, partner))\n", + " return pairs\n", + "\n", + "\n", + "def sample_association_local_distances(\n", + " embeddings: np.ndarray,\n", + " gt_ids: np.ndarray,\n", + " *,\n", + " frame_ids: np.ndarray,\n", + " seq_ids: np.ndarray,\n", + " n_intra: int = N_INTRA,\n", + " n_inter: int = N_INTER,\n", + " min_frame_gap: int = MIN_FRAME_GAP,\n", + " max_frame_gap: int = MAX_FRAME_GAP,\n", + " seed: int = 0,\n", + ") -> tuple[np.ndarray, np.ndarray]:\n", + " \"\"\"Same-video pairs within max_frame_gap (tracker association horizon).\n", + "\n", + " Pairs are drawn directly rather than enumerated into a pool, and every sequence\n", + " gets the same quota, so no single crowded sequence can decide the histogram.\n", + " Same-ID pairs pick an identity uniformly so long tracks do not dominate.\n", + " \"\"\"\n", + " if min_frame_gap < 1:\n", + " raise ValueError(\"min_frame_gap must be >= 1, otherwise a crop can pair with itself\")\n", + " normed = embeddings / (np.linalg.norm(embeddings, axis=1, keepdims=True) + 1e-12)\n", + " rng = np.random.default_rng(seed)\n", + "\n", + " # Per sequence: crop indexes sorted by frame, their frames and ids, and the\n", + " # slot lists of every identity seen more than once.\n", + " per_seq: dict[int, tuple] = {}\n", + " for sid in np.unique(seq_ids):\n", + " order = np.flatnonzero(seq_ids == sid)\n", + " order = order[np.argsort(frame_ids[order], kind=\"stable\")]\n", + " by_id: dict[int, list[int]] = defaultdict(list)\n", + " for slot, idx in enumerate(order):\n", + " by_id[int(gt_ids[idx])].append(slot)\n", + " tracks = [np.asarray(v) for v in by_id.values() if len(v) > 1]\n", + " per_seq[int(sid)] = (order, frame_ids[order], gt_ids[order], tracks)\n", + "\n", + " sampled: list[np.ndarray] = []\n", + " sids = sorted(per_seq)\n", + " for quota, same_id in ((n_intra, True), (n_inter, False)):\n", + " pairs: list[tuple[int, int]] = []\n", + " for k, sid in enumerate(sids):\n", + " order, frames, ids, tracks = per_seq[sid]\n", + " local = _draw_pairs_in_band(\n", + " rng,\n", + " frames=frames,\n", + " ids=ids,\n", + " tracks=tracks,\n", + " n_wanted=quota // len(sids) + (1 if k < quota % len(sids) else 0),\n", + " same_id=same_id,\n", + " min_frame_gap=min_frame_gap,\n", + " max_frame_gap=max_frame_gap,\n", + " )\n", + " pairs.extend((int(order[a]), int(order[b])) for a, b in local)\n", + " if not pairs:\n", + " kind = \"same-ID\" if same_id else \"different-ID\"\n", + " raise ValueError(f\"No {kind} pairs with {min_frame_gap}<=|Delta frame|<={max_frame_gap}.\")\n", + " sampled.append(_pair_distances(normed, np.asarray(pairs, dtype=np.int64)))\n", + " return sampled[0], sampled[1]\n", + "\n", + "\n", + "emb_gt, gt_ids, frame_ids, seq_ids = collect_gt_embeddings(reid_model, ACTIVE_SEQUENCES)\n", + "print(f\"GT pool: {len(emb_gt)} crops, {len(np.unique(gt_ids))} ids, {len(np.unique(seq_ids))} sequences\")\n", + "\n", + "intra, inter = sample_association_local_distances(\n", + " emb_gt, gt_ids, frame_ids=frame_ids, seq_ids=seq_ids\n", + ")\n", + "print(f\"pairs: same-ID={len(intra)} diff-ID={len(inter)} (same seq, |Delta frame|<={MAX_FRAME_GAP})\")\n", + "print(\n", + " f\"d_app means: same-ID={intra.mean():.3f} diff-ID={inter.mean():.3f} \"\n", + " f\"gap={inter.mean() - intra.mean():.3f} \"\n", + " f\"same-ID p95={np.quantile(intra, 0.95):.3f}\"\n", + ")\n", + "\n", + "fig, ax = plt.subplots(figsize=(8, 4.5))\n", + "for values, label, color in (\n", + " (intra, f\"same-ID (n={len(intra)})\", \"#3366CC\"),\n", + " (inter, f\"different-ID (n={len(inter)})\", \"#DC3912\"),\n", + "):\n", + " ax.hist(values, bins=DAPP_BINS, weights=np.full(len(values), 1.0 / len(values)), alpha=0.65, label=label, color=color)\n", + "ax.axvline(0.25, color=\"#666666\", ls=\":\", lw=1.5, label=\"θ=0.25 (default)\")\n", + "ax.axvline(REID_APPEARANCE_THRESHOLD, color=\"#111111\", ls=\"--\", lw=1.8, label=f\"θ={REID_APPEARANCE_THRESHOLD:.2f} (selected)\")\n", + "ax.set(xlabel=r\"$0.5\\cdot$ cosine distance\", ylabel=\"probability\", title=f\"{REID_ENCODER} on MOT17 val GT\", xlim=(0.0, 0.6))\n", + "ax.legend(frameon=False, fontsize=9)\n", + "ax.grid(True, alpha=0.25)\n", + "fig.tight_layout()\n", + "plt.show()\n", + "\n", + "print(f\"{'θ':>6} {'same-ID < θ':>12} {'diff-ID < θ':>12}\")\n", + "for theta in CANDIDATE_THETAS:\n", + " note = \" <- selected\" if abs(theta - REID_APPEARANCE_THRESHOLD) < 1e-9 else (\" <- default\" if abs(theta - 0.25) < 1e-9 else \"\")\n", + " print(f\"{theta:6.2f} {100 * float(np.mean(intra < theta)):11.1f}% {100 * float(np.mean(inter < theta)):11.1f}%{note}\")\n", + "" + ], + "execution_count": null, + "outputs": [], + "id": "b2e224c9" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## How far the threshold carries\n", + "\n", + "The histogram above fixes the frame gap at `MAX_FRAME_GAP`, so it only describes\n", + "re-association over that horizon. Sweeping the gap shows how long a track can stay\n", + "lost before appearance stops helping to re-find it.\n", + "\n", + "ROC AUC is the chance that a random same-ID pair scores closer than a random\n", + "different-ID pair, so its complement is how often a same-ID pair sits farther apart\n", + "than a different-ID one. It is the area under the curve you get by sweeping θ from 0\n", + "to 1 and plotting the two printed rates, which is why it summarises every threshold\n", + "instead of the single one you picked.\n", + "\n", + "It is not the area where the shaded bands cross: that is two percentile ranges\n", + "intersecting, which ignores where the mass sits and which side is closer. At a\n", + "1-frame gap the bands never touch yet the AUC is 0.998 rather than 1.0." + ], + "id": "ddd6422c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "GAP_BUCKETS = [(1, 1), (2, 5), (6, 15), (16, 30), (31, 60), (61, 120), (121, 240)]\n", + "N_SWEEP_PER_CLASS = 4000\n", + "\n", + "\n", + "def roc_auc(intra: np.ndarray, inter: np.ndarray) -> float:\n", + " \"\"\"P(same-ID distance < different-ID distance), ties counted as half.\"\"\"\n", + " inter_sorted = np.sort(inter)\n", + " right = np.searchsorted(inter_sorted, intra, side=\"right\")\n", + " left = np.searchsorted(inter_sorted, intra, side=\"left\")\n", + " return float(np.mean(((len(inter) - right) + 0.5 * (right - left)) / len(inter)))\n", + "\n", + "\n", + "sweep: list[tuple[str, np.ndarray, np.ndarray]] = []\n", + "for lo, hi in GAP_BUCKETS:\n", + " try:\n", + " gap_intra, gap_inter = sample_association_local_distances(\n", + " emb_gt,\n", + " gt_ids,\n", + " frame_ids=frame_ids,\n", + " seq_ids=seq_ids,\n", + " n_intra=N_SWEEP_PER_CLASS,\n", + " n_inter=N_SWEEP_PER_CLASS,\n", + " min_frame_gap=lo,\n", + " max_frame_gap=hi,\n", + " )\n", + " except ValueError:\n", + " print(f\"gap {lo}-{hi}: no pairs, skipped\")\n", + " continue\n", + " sweep.append((str(lo) if lo == hi else f\"{lo}-{hi}\", gap_intra, gap_inter))\n", + "\n", + "x = np.arange(len(sweep))\n", + "LO_PCT, HI_PCT = 10, 90 # symmetric band, so both classes are read the same way\n", + "intra_q = np.array([np.percentile(row[1], [LO_PCT, 50, HI_PCT]) for row in sweep])\n", + "inter_q = np.array([np.percentile(row[2], [LO_PCT, 50, HI_PCT]) for row in sweep])\n", + "\n", + "fig, (ax_dist, ax_auc) = plt.subplots(2, 1, figsize=(8, 6.5), sharex=True, gridspec_kw={\"height_ratios\": [2.2, 1.0]})\n", + "ax_dist.fill_between(x, intra_q[:, 0], intra_q[:, 2], color=\"#3366CC\", alpha=0.22)\n", + "ax_dist.plot(x, intra_q[:, 1], color=\"#3366CC\", marker=\"o\", lw=2, label=\"same ID\")\n", + "ax_dist.fill_between(x, inter_q[:, 0], inter_q[:, 2], color=\"#DC3912\", alpha=0.22)\n", + "ax_dist.plot(x, inter_q[:, 1], color=\"#DC3912\", marker=\"o\", lw=2, label=\"different ID\")\n", + "ax_dist.axhline(\n", + " REID_APPEARANCE_THRESHOLD,\n", + " color=\"#111111\",\n", + " ls=\"--\",\n", + " lw=1.5,\n", + " label=f\"θ = {REID_APPEARANCE_THRESHOLD:.2f} (selected)\",\n", + ")\n", + "ax_dist.axhline(0.25, color=\"#666666\", ls=\":\", lw=1.5, label=\"θ = 0.25 (default)\")\n", + "ax_dist.set(ylabel=\"appearance distance\")\n", + "ax_dist.set_title(f\"line = median, shaded = {LO_PCT}th to {HI_PCT}th percentile\", fontsize=8.5, color=\"#333333\", pad=4)\n", + "fig.suptitle(f\"{REID_ENCODER}: separability vs frame gap\", y=0.995)\n", + "ax_dist.legend(loc=\"lower right\", fontsize=9, ncol=2, framealpha=0.92, edgecolor=\"none\")\n", + "ax_dist.grid(True, alpha=0.25)\n", + "\n", + "aucs = [roc_auc(row[1], row[2]) for row in sweep]\n", + "ax_auc.plot(x, aucs, color=\"#111111\", marker=\"s\", lw=2)\n", + "for xi, auc in zip(x, aucs):\n", + " ax_auc.annotate(f\"{auc:.3f}\", (xi, auc), textcoords=\"offset points\", xytext=(0, 7), ha=\"center\", fontsize=7.5)\n", + "ax_auc.axhline(0.5, color=\"#999999\", ls=\":\", lw=1.2)\n", + "ax_auc.set(\n", + " xlabel=\"frames between the two crops\",\n", + " ylabel=\"separability\",\n", + " ylim=(0.42, 1.12),\n", + " xticks=x,\n", + " xticklabels=[row[0] for row in sweep],\n", + ")\n", + "ax_auc.set_title(\n", + " \"take one same-ID and one different-ID pair at random: how often is the same-ID one closer?\"\n", + " \"\\n1.0 = always, 0.5 = coin flip. Counts every sampled pair, not the shaded overlap above.\",\n", + " fontsize=8,\n", + " color=\"#333333\",\n", + " pad=4,\n", + ")\n", + "ax_auc.grid(True, alpha=0.25)\n", + "fig.tight_layout()\n", + "plt.show()\n", + "\n", + "print(f\"{'gap':>10} {'AUC':>6} {'same-ID < θ':>12} {'diff-ID < θ':>12}\")\n", + "for label, gap_intra, gap_inter in sweep:\n", + " print(\n", + " f\"{label:>10} {roc_auc(gap_intra, gap_inter):6.3f} \"\n", + " f\"{100 * np.mean(gap_intra < REID_APPEARANCE_THRESHOLD):11.1f}% \"\n", + " f\"{100 * np.mean(gap_inter < REID_APPEARANCE_THRESHOLD):11.1f}%\"\n", + " )" + ], + "execution_count": null, + "outputs": [], + "id": "fff63fa6" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Results\n", + "\n", + "Score BoT-SORT + ReID on MOT17 val-half. Reference numbers below come from the\n", + "[MOT17 re-ID study](https://www-sop.inria.fr/members/Francois.Bremond/Postscript/Tomasz__SCCAI_2025.pdf)\n", + "(Tables 8 and 13) and the [BoT-SORT paper](https://arxiv.org/abs/2206.14651) (Table 1).\n", + "\n", + "| Source | Config | HOTA | IDF1 |\n", + "|---|---|---:|---:|\n", + "| MOT17 re-ID study | No re-ID | 68.43 | 80.92 |\n", + "| MOT17 re-ID study | FastReID, th=0.2 | 68.95 | 81.98 |\n", + "| BoT-SORT paper | BoT-SORT | 69.11 | 81.53 |\n", + "| BoT-SORT paper | BoT-SORT + ReID | 69.17 | 82.07 |\n" + ], + "id": "results-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "REF_NO_REID = {\"hota\": 68.43, \"idf1\": 80.92}\n", + "REF_REID = {\"hota\": 68.95, \"idf1\": 81.98}\n", + "\n", + "agg = result_reid.aggregate\n", + "hota = agg.HOTA.HOTA * 100\n", + "mota = agg.CLEAR.MOTA * 100\n", + "idf1 = agg.Identity.IDF1 * 100\n", + "idsw = agg.CLEAR.IDSW\n", + "\n", + "print(f\"{'Config':<28} {'HOTA':>6} {'MOTA':>6} {'IDF1':>6} {'IDSW':>5}\")\n", + "print(\"-\" * 58)\n", + "print(f\"{'BoT-SORT + ReID (this run)':<28} {hota:6.2f} {mota:6.2f} {idf1:6.2f} {idsw:5d}\")\n", + "print(f\"{'MOT17 study (no ReID)':<28} {REF_NO_REID['hota']:6.2f} {'-':>6} {REF_NO_REID['idf1']:6.2f} {'-':>5}\")\n", + "print(f\"{'MOT17 study (FastReID)':<28} {REF_REID['hota']:6.2f} {'-':>6} {REF_REID['idf1']:6.2f} {'-':>5}\")\n", + "print(\n", + " f\"\\nvs MOT17 study FastReID: \"\n", + " f\"HOTA {hota - REF_REID['hota']:+.2f}, IDF1 {idf1 - REF_REID['idf1']:+.2f}\"\n", + ")\n", + "\n", + "print(f\"\\n{'Sequence':<18} {'HOTA':>6} {'AssA':>6} {'IDF1':>6} {'IDSW':>5}\")\n", + "print(\"-\" * 48)\n", + "for seq in ACTIVE_SEQUENCES:\n", + " s = result_reid.sequences[seq]\n", + " print(\n", + " f\"{seq:<18} {s.HOTA.HOTA * 100:6.2f} {s.HOTA.AssA * 100:6.2f} \"\n", + " f\"{s.Identity.IDF1 * 100:6.2f} {s.CLEAR.IDSW:5d}\"\n", + " )\n" + ], + "execution_count": null, + "outputs": [], + "id": "results-code" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Sample tracked frames\n", + "\n", + "Plot a few frames from one sequence with BoT-SORT + ReID track IDs.\n" + ], + "id": "viz-md" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "VIZ_SEQ = ACTIVE_SEQUENCES[0] # override with e.g. \"MOT17-02-FRCNN\"\n", + "VIZ_FRAMES = (1, 30, 60, 90)\n", + "\n", + "mot = load_mot_file(pred_dir / f\"{VIZ_SEQ}.txt\")\n", + "img_dir = SEQUENCE_PATHS[VIZ_SEQ][\"img\"]\n", + "box_ann = sv.BoxAnnotator(thickness=2, color_lookup=sv.ColorLookup.TRACK)\n", + "label_ann = sv.LabelAnnotator(\n", + " text_color=sv.Color.BLACK,\n", + " text_scale=0.5,\n", + " color_lookup=sv.ColorLookup.TRACK,\n", + ")\n", + "\n", + "fig, axes = plt.subplots(2, 2, figsize=(12, 8))\n", + "for ax, frame_idx in zip(axes.ravel(), VIZ_FRAMES):\n", + " frame = load_mot_frame_image(img_dir, frame_idx)\n", + " data = mot.get(frame_idx)\n", + " if data is None or len(data.ids) == 0:\n", + " scene = frame\n", + " else:\n", + " dets = sv.Detections(\n", + " xyxy=sv.xywh_to_xyxy(data.boxes).astype(np.float32),\n", + " tracker_id=data.ids.astype(int),\n", + " )\n", + " scene = box_ann.annotate(frame.copy(), dets)\n", + " scene = label_ann.annotate(scene, dets, labels=[str(int(i)) for i in dets.tracker_id])\n", + " ax.imshow(scene[:, :, ::-1])\n", + " ax.set_title(f\"{VIZ_SEQ} frame {frame_idx}\")\n", + " ax.axis(\"off\")\n", + "\n", + "fig.suptitle(\"BoT-SORT + ReID\", y=1.01)\n", + "fig.tight_layout()\n", + "plt.show()" + ], + "execution_count": null, + "outputs": [], + "id": "viz-code" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You just ran BoT-SORT with appearance ReID on MOT17. Nice work!\n", + "\n", + "Trackers makes it easy to mix and match multi-object tracking algorithms with your\n", + "favorite detection backends. Appearance association is optional: install\n", + "`trackers[reid]`, pass a `reid.ReIDModel`, and supply `frame=` to `update()`.\n", + "\n", + "Ready to go deeper? Explore the [ReID appearance guide](https://trackers.roboflow.com/latest/learn/reid/),\n", + "the [`reid` package](https://github.com/roboflow/re-ID), or the Trackers\n", + "[documentation](https://trackers.roboflow.com/latest/) and\n", + "[GitHub](https://github.com/roboflow/trackers).\n", + "\n", + "Got feedback or ideas? Open an issue on\n", + "[GitHub Issues](https://github.com/roboflow/trackers/issues)." + ], + "id": "outro" + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file