Skip to content
This repository was archived by the owner on Jun 3, 2026. It is now read-only.

docs: add chaos testing doc and example script#836

Open
ybdarrenwang wants to merge 6 commits into
strands-agents:mainfrom
ybdarrenwang:docs/chaos-tool
Open

docs: add chaos testing doc and example script#836
ybdarrenwang wants to merge 6 commits into
strands-agents:mainfrom
ybdarrenwang:docs/chaos-tool

Conversation

@ybdarrenwang
Copy link
Copy Markdown
Contributor

@ybdarrenwang ybdarrenwang commented May 13, 2026

Description

Adds documentation and example for the chaos testing module in Strands Evals.

The example demonstrates:

  • How chaos testing would impact GoalSuccessRateEvaluator and how to evaluate agent resilience with 3 new chaos evaluators (FailureCommunicationEvaluator, PartialCompletionEvaluator, RecoveryStrategyEvaluator)
  • Defining named effect maps (dict[str, dict[str, list[ChaosEffect]]]) for readable failure conditions (e.g., "search_timeout", "total_chaos")
  • Using ChaosCase.expand(cases, effect_maps) to generate the Cartesian product of test cases × failure conditions with optional baseline
  • Running a ChaosExperiment with ChaosPlugin for transparent fault injection

The example uses ToolSimulator for reproducible tool responses and covers pre-hook effects (tool call failures), post-hook effects (response corruption), and compound multi-tool chaos.

Related Issues

#114

Type of Change

  • New content

Checklist

  • I have read the CONTRIBUTING document
  • My changes follow the project's documentation style
  • I have tested the documentation locally using npm run dev
  • Links in the documentation are valid and working

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@ybdarrenwang ybdarrenwang changed the title docs: add chaos testing example script docs: add chaos testing doc and example script May 29, 2026
Comment thread site/docs/examples/evals-sdk/chaos_failure_communication.py Outdated
@strands-agent
Copy link
Copy Markdown
Contributor

This repository has been merged into the strands-agents/harness-sdk monorepo and will be archived shortly. All new development happens there.

If this PR is still relevant, please recreate it against the monorepo. The code now lives under site/. Full commit history was preserved, so your base should be findable.

Apologies for the disruption, and thank you for contributing!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants