Skip to content

Add random_state to DummyOutcomeRefuter for reproducible refutations#1557

Merged
amit-sharma merged 1 commit into
py-why:mainfrom
immu4989:fix/dummy-outcome-refuter-random-state
Jun 6, 2026
Merged

Add random_state to DummyOutcomeRefuter for reproducible refutations#1557
amit-sharma merged 1 commit into
py-why:mainfrom
immu4989:fix/dummy-outcome-refuter-random-state

Conversation

@immu4989
Copy link
Copy Markdown
Contributor

Follow-up to #1556, part of making DoWhy's refutation results reproducible (related: #556, #418).

What

DummyOutcomeRefuter is the only refuter whose randomness cannot be seeded. refute_dummy_outcome, _refute_once, process_data, noise, and permute all draw from the global numpy random state, and the train/validation split uses an unseeded DataFrame.sample(). As a result, running the dummy-outcome refutation twice on the same data yields different results.

The sibling refuters — BootstrapRefuter, DataSubsetRefuter, PlaceboTreatmentRefuter, and RandomCommonCause — already expose a random_state parameter. This PR brings DummyOutcomeRefuter in line with that established API.

Changes

  • Add random_state to DummyOutcomeRefuter and thread it through refute_dummy_outcome_refute_onceprocess_datanoise/permute, plus the train/validation DataFrame.sample() calls.
  • Follow the same convention as RandomCommonCause: accept an int or np.random.RandomState, convert once, and pass it through joblib.Parallel.
  • Default (random_state=None) preserves existing behavior, so this is backward compatible.
  • Add regression tests asserting that same-seed runs are identical and different-seed runs differ.

Verification

  • Same seed → identical refutation values; different seeds → different values; default path unchanged.
  • New tests in tests/causal_refuters/test_dummy_outcome_refuter.py pass.

Signed-off-by: Imran Ahamed <immu4989@gmail.com>
Copy link
Copy Markdown
Member

@amit-sharma amit-sharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this looks good

@amit-sharma amit-sharma merged commit 62159a3 into py-why:main Jun 6, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants