Skip to content

Fix destination reachability and add shared-basis evaluation metrics - #3

Merged
sergio-gimenez merged 5 commits into
mainfrom
fix/shared-reachability-basis
Sep 4, 2026
Merged

Fix destination reachability and add shared-basis evaluation metrics#3
sergio-gimenez merged 5 commits into
mainfrom
fix/shared-reachability-basis

Conversation

@sergio-gimenez

Copy link
Copy Markdown
Collaborator

Summary

Two coupled problems fixed, plus tooling for running the evaluation campaign:

  1. Link-state could not reach deliverable ground stations. The Hypatia-derived fstate path collapsed per-GS visibility to the single nearest satellite before routing, which made the destination a fixed satellite rather than the ground station itself. Under sparse connectivity that satellite is frequently in an unreachable component while another visible one is reachable, so link-state reported failure on pairs that were deliverable — in a Ring topology it delivered 80 of 408 deliverable pairs instead of all of them.
  2. The stretch baseline was algorithm-dependent. Each algorithm was graded against a shortest path to whichever egress satellite it happened to reach, over whichever subset of pairs it managed to deliver. Neither the subset nor its size was reported, so an algorithm that fails on the hard pairs was rewarded: only its successes reached the average.

Changes

Routing fix (6678dc6)

  • Thread the full per-GS visibility list (ground_station_satellites_in_range) through algorithm_free_one_only_over_isls and the fstate calculation, so the destination egress is whichever visible satellite minimises path length plus GSL length.
  • Single-attachment fallback retained for callers without a precomputed visibility list; it understates reachability and logs as such.
  • Test call sites updated for the new parameter.

Metrics (f669f87)

  • Reachability is now decided from the topology alone, before any algorithm is consulted, via _best_reachable_egress: the best end-to-end route to any satellite the destination can see (hop-optimal and distance-optimal egress tracked separately, since they need not coincide). Every algorithm is scored over the same pairs against the same lower bound.
  • New delivery_* columns: deliverable pairs, delivered pairs, delivery rate, forwarding failures, and the separate causes of non-delivery (no_src_visibility, no_dst_visibility, disconnected).
  • New stretch_hop_shared / stretch_dist_shared graded against the shared lower bound; legacy stretch_hop / stretch_dist kept for continuity with earlier runs.
  • New delivery_non_optimal_egress_rate: share of delivered pairs exiting through a non-optimal egress — the quantity that separates the two stretch families.
  • Semantics documented in docs/evaluation.md.

Tooling (03e7959)

  • scripts/run-matrix-parallel.sh: config × algorithm × ISL-scenario matrix as parallel Docker jobs, JOBS-bounded, BLAS pinned to 1 thread per container, completed jobs skipped on rerun (interrupted campaigns resume), per-job wall-clock and exit code in job_timings.csv.

Verification

  • tests/network_state/test_algorithm_free_one_only_over_isls.py: 4 passed.
  • By construction, a shortest-path algorithm scores 1.0 on the shared basis — link-state doubles as a correctness check on the metric itself.

Chores

  • dbbed6a backfills the CHANGELOG Unreleased section for the baseline-removal refactor (ad05c47, previously unrecorded) and this branch.

The Hypatia-derived fstate path collapsed per-GS visibility to the
single nearest satellite before routing, which made the destination a
fixed satellite rather than the ground station itself. Under sparse
connectivity that satellite is frequently in an unreachable component
while another visible one is reachable, so link-state reported failure
on deliverable pairs: in a Ring topology it delivered 80 of 408
deliverable pairs instead of all of them.

Thread the full per-GS visibility list (ground_station_satellites_in_range)
through algorithm_free_one_only_over_isls and the fstate calculation so
the destination egress is whichever visible satellite minimises path
length plus GSL length. The single-attachment fallback is kept for
callers without a precomputed visibility list; it understates
reachability and is logged as such.
The previous stretch baseline was a shortest path to whichever egress
satellite the algorithm being measured happened to reach, so each
algorithm was graded against a different target over a different subset
of pairs, and neither the subset nor its size was reported. An algorithm
that fails on hard pairs was rewarded, because only its successes
reached the average.

Compute reachability from the topology before any algorithm is
consulted, via _best_reachable_egress: the best end-to-end route to any
satellite the destination can see, hop-optimal and distance-optimal
egress tracked separately since they need not coincide. Every algorithm
is now scored over the same pairs against the same lower bound.

- delivery_* columns: deliverable pairs, delivered pairs, delivery
  rate, forwarding failures, and non-delivery causes (no source
  visibility, no destination visibility, graph disconnection)
- stretch_hop_shared / stretch_dist_shared graded against the shared
  lower bound; legacy stretch_hop / stretch_dist kept for continuity
- delivery_non_optimal_egress_rate: share of delivered pairs exiting
  through a non-optimal egress, what separates the two families

A shortest-path algorithm scores 1.0 on the shared basis by
construction, so link-state doubles as a correctness check on the
metric. Metric semantics documented in docs/evaluation.md.
Runs a config x algorithm x ISL-scenario matrix as parallel docker
jobs, one single-threaded python process per job with concurrency
bounded by JOBS. BLAS threading is pinned to 1 inside the container:
without that numpy oversubscribes and 24 jobs fight over the machine.

- skips jobs whose timestep_metrics.csv and metadata.json already
  exist, so an interrupted campaign resumes by rerunning the command
- appends per-job wall-clock and exit code to job_timings.csv
- heaviest constellations enqueued first so the long pole starts early
- per-algorithm flags (distance mode, segment refresh/egress mode)
  resolved from env with the defaults used by the paper runs

  JOBS=24 ./scripts/run-matrix-parallel.sh /path/to/output
Backfill the Unreleased section for the baseline-removal refactor and
the changes on this branch: the any-visible-satellite routing fix, the
shared stretch basis and delivery accounting, and the parallel matrix
runner.
CI lint runs black --check on leopath and tests. Two files drifted:

- plot_seam_robustness.py: pre-existing since main; old magic-trailing-
  comma style on two multi-line calls
- metrics.py: the parenthesised _best_reachable_egress call and the
  non_optimal_egress_rate entry from this branch collapsed differently
  than black 25.x prefers now that the expressions fit on one line

No semantic changes; 147 tests pass.
@sergio-gimenez
sergio-gimenez merged commit 99b836d into main Sep 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant