Skip to content

EXPERIMENT_CONTRACT.md

Latest commit

 

History

History
53 lines (42 loc) · 3.12 KB

EXPERIMENT_CONTRACT.md

File metadata and controls

53 lines (42 loc) · 3.12 KB
Raw

Experiment contract

Every folder under experiments/ is an independent, publishable unit: own uv env, own README written as a short paper, own reproducible run.py, own tests. Shared code lives only in qcore (path dependency).

Layout

experiments/NN-slug/
  .python-version        # 3.13
  pyproject.toml         # deps: qcore (editable path) + what this experiment needs
  README.md              # paper-style (see template below)
  run.py                 # `uv run python run.py [--quick] [--backend X] [--seed N] [--out results]`
  src/<pkg>/             # importable code, no notebooks
  tests/                 # `uv run pytest -q` must pass in < 60 s
  results/               # generated: results.json, *.png, *.csv (committed for the record)

Rules

  1. uv run python run.py --quick finishes in < 2 min on a laptop; full run documented with wall-clock.
  2. All randomness seeded (--seed). results/results.json includes {"seed", "backend", "qcore_version", "timestamp", "config", "metrics"}.
  3. Backend chosen via qcore.get_backend(args.backend); never import a simulator directly except for features qcore lacks (state that in README).
  4. Always compare against ≥ 1 classical baseline and report honestly (no "quantum beats classical" unless statistically supported with CI).
  5. Every quantity in README "Results" section is produced by run.py (copy from results.json).
  6. Tests: unit tests on the problem encoding (small brute-force checks) + a --quick smoke run.
  7. No global side effects: no network downloads at import time; datasets are synthetic or generated by a script in src/<pkg>/data.py with citation.
  8. Before writing §5/§6 of the README, go through the 10-point checklist in docs/04-cross-experiment-findings.md §5 (strongest baseline incl. a local search on the real objective; proxy-vs-real objective audit near the optimum; feasibility of the exact optimum and raw-vs-repaired sample quality; inherited classical inputs incl. horizons and surrogate training sets; lossy pre-processing; simulability; seeds + CIs + a same-budget structure-removing control; equal evaluation budgets vs random search, including a uniform sampler at the same shot budget through the same decode/repair, seeded per instance; legality checks behind structural claims; the classical solver's exact-size limit and time).

README template (paper structure)

# <Title>

**Abstract.** 4–6 sentences: problem, encoding, quantum primitive, what was measured, main finding.

## 1. Background & gap          (what exists, what nobody did — cite arXiv IDs)
## 2. Problem formulation       (math; QUBO / Hamiltonian / oracle definition)
## 3. Method                    (circuit / algorithm; classical baselines; metrics)
## 4. Experimental setup        (sizes, seeds, shots, backend, hardware, wall-clock)
## 5. Results                   (tables from results.json + figures)
## 6. Discussion & limitations  (honest; where advantage is / isn't; scaling)
## 7. Reproduce                 (`uv sync && uv run python run.py`)
## References

Setup snippet

cd experiments/NN-slug
uv sync
uv run pytest -q
uv run python run.py --quick