Skip to content

Add random_state to CausalEstimator for reproducible bootstrap CIs and significance tests#1556

Open
immu4989 wants to merge 2 commits into
py-why:mainfrom
immu4989:fix/estimator-bootstrap-random-state
Open

Add random_state to CausalEstimator for reproducible bootstrap CIs and significance tests#1556
immu4989 wants to merge 2 commits into
py-why:mainfrom
immu4989:fix/estimator-bootstrap-random-state

Conversation

@immu4989
Copy link
Copy Markdown
Contributor

@immu4989 immu4989 commented May 31, 2026

Closes #418
Closes #556

What

Bootstrap confidence intervals and significance tests in the shared CausalEstimator base class are non-deterministic, because they draw from the global numpy random state:

  • _generate_bootstrap_estimates calls sklearn.utils.resample(...) with no random_state, so bootstrap CIs differ between runs.
  • _test_significance_with_bootstrap calls np.random.permutation(...), so bootstrap p-values differ between runs.

This affects every estimator that uses bootstrap CIs or significance testing.

Changes

  • Add a random_state parameter (int or np.random.RandomState, default None) to CausalEstimator.
  • Add _get_random_state() and use it to seed both resample(...) and the permutation in the significance test.
  • random_state=None preserves the existing (non-deterministic) behavior, so this is backward compatible.
  • Only the resampling layers are seeded; a user-supplied propensity_score_model is left untouched.
  • Add regression tests asserting that two runs with the same random_state produce identical confidence intervals and p-values, while different seeds differ.

Usage

estimate = model.estimate_effect(
    identified_estimand,
    method_name="backdoor.propensity_score_weighting",
    confidence_intervals=True,
    test_significance=True,
    method_params={"init_params": {"random_state": 42}},
)

Verification

  • Same seed → identical CIs and p-values; different seeds → different CIs.
  • Existing estimator tests pass; default behavior unchanged.

…d significance tests

Signed-off-by: Imran Ahamed <immu4989@gmail.com>
Signed-off-by: Amit Sharma <amit_sharma@live.com>
Copy link
Copy Markdown
Member

@amit-sharma amit-sharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this fix @immu4989 . Much appreciated

@amit-sharma
Copy link
Copy Markdown
Member

@all-contributors please add @immu4989 for code

@allcontributors
Copy link
Copy Markdown
Contributor

@amit-sharma

I've put up a pull request to add @immu4989! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How can i get a deterministic output/estimate? Setting the seed for estimate effect methods

2 participants