fuzz: dictionary-guided IPC fuzzing for CI#11004
Open
tmleman wants to merge 4 commits into
Open
Conversation
tmleman
requested review from
dbaluta,
kv2019i,
lbetlej,
lgirdwood,
mmaka1 and
plbossart
as code owners
July 16, 2026 12:42
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves CI fuzzing effectiveness for SOF IPC3/IPC4 by generating libFuzzer dictionaries directly from in-tree IPC headers (to avoid drift) and wiring dictionary use into the existing fuzzing workflow in a best-effort way (non-fatal if generation fails).
Changes:
- Add two Python generators to harvest IPC3/IPC4 dispatch constants/enums from headers and emit libFuzzer dictionary files.
- Extend
scripts/fuzz.shwith an optional-d <dict>passthrough to run with-dict=...when present/non-empty. - Update the GitHub Actions IPC fuzzing workflow to regenerate a temporary dictionary per run and pass it into
fuzz.sh.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
scripts/gen_fuzz_ipc4_dict.py |
New IPC4 header-synchronized dictionary generator for libFuzzer. |
scripts/gen_fuzz_ipc3_dict.py |
New IPC3 header-synchronized dictionary generator for libFuzzer. |
scripts/fuzz.sh |
Adds optional -d handling to pass a dictionary to libFuzzer with non-fatal fallback. |
.github/workflows/ipc_fuzzer.yml |
Regenerates per-run IPC dictionaries in CI and passes them to the fuzzer run. |
| # inject known byte sequences into the candidate inputs. For SOF | ||
| # IPC4 the most valuable seed values are: | ||
| # | ||
| # * The 4-byte "primary" message header dat for each well-known |
| # (pipeline state values, pipeline extension object IDs, | ||
| # large_config param IDs). | ||
| # | ||
| # The constants are harvested directly from sof/src/include/ipc4/*.h |
Comment on lines
+103
to
+105
| def harvest(repo_root): | ||
| """Parse header.h, returning (glb_defs, cmd_defs) name->value dicts.""" | ||
| path = repo_root / HEADER_REL |
| if not path.is_file(): | ||
| print(f"error: {path} not found", file=sys.stderr) | ||
| sys.exit(1) | ||
| text = path.read_text() |
Comment on lines
+201
to
+203
| if args.output: | ||
| Path(args.output).write_text(text) | ||
| else: |
| if not path.is_file(): | ||
| print(f"error: {path} not found", file=sys.stderr) | ||
| sys.exit(1) | ||
| text = path.read_text() |
Comment on lines
+261
to
+263
| if args.output: | ||
| Path(args.output).write_text(text) | ||
| else: |
The native_sim libFuzzer harness consults an external dictionary (`-dict=`) for known byte sequences to splice into mutated inputs. Until now the only dictionary content shipped for the IPC4 target was two placeholder entries, leaving libFuzzer to discover the whole IPC4 dispatch surface (global / module message types, pipeline states, large-config param IDs) on its own through random mutation plus CMP intercepts. Most random 4-byte prefixes are rejected by the type-field switch in the IPC4 front end, so the deeper handlers were reached only rarely. Add a generator that harvests integer-literal enums from src/include/ipc4/*.h and emits a libFuzzer dictionary mapping each constant to a 4-byte little-endian token, with two specialised encodings for dispatch headers: * global_pri - (type << 24) (msg_tgt=0, rsp=0) * module_pri - (1 << 30) | (type << 24) (msg_tgt=1) so that every well-known global or module entry point appears as a ready-to-paste 4-byte pri value at any 4-byte-aligned offset chosen by the mutator. Raw u32 encodings are emitted for the remaining enums (pipeline state values, pipeline extension object IDs, base-fw / hw-config / memory-type / clock-src params, notification and resource types, error/status codes). The `type` field of the primary header is only 5 bits wide, so pri tokens are emitted only for enum values that fit (0..31); MAX / COUNT sentinels are dropped by value as well as by name, so a header typo fix cannot leak a bogus dispatch token. The parser is intentionally narrow: it only follows enums whose entries are plain integer literals (with optional auto-increment), stopping at the first non-literal initialiser. That keeps the script free of preprocessor logic while capturing the great majority of dispatch-affecting constants; enums that use cross-references or bit-field expressions can be hand-added to the ENUM_SOURCES table when worthwhile. A missing header or an empty harvest is a hard error, so a broken regeneration can never silently overwrite a good dictionary with an empty one. Usage: python3 scripts/gen_fuzz_ipc4_dict.py -o <output.dict> Coverage impact, measured on the coverage-instrumented harness (IPC4, empty corpus, 5 seeds x 60 s, paired dict-minus-nodict means): libFuzzer edges (cov) +69 (high variance, seed dependent) libFuzzer features (ft) +141 IPC subsystem lines +23 (+0.9 pp of src/ipc) The dictionary's main effect is to raise the coverage ceiling: it occasionally unlocks deep module-dispatch paths that random mutation rarely reaches (best-seed firmware line coverage 2122 -> 3268), so the per-run gain is positive on average but noisy at 60 s. The output is not committed; it is regenerated from the headers on demand (for example by CI into a temporary file) and deployments may extend it with site-specific tokens. Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
The native_sim libFuzzer harness consults an external dictionary (`-dict=`) for known byte sequences to splice into mutated inputs. The IPC3 dictionary shipped until now was hand-written: thirteen opaque `kwN=` entries, two of which were corpus-derived byte blobs rather than real dispatch constants, covering only a subset of the topology / PM / stream command space. It had no documented provenance and silently drifted out of sync as header.h changed. Add a generator that harvests the SOF_GLB_TYPE() / SOF_CMD_TYPE() combines the global type with its command type: cmd = (glb_type << 28) | (cmd_type << 16) so each token lands directly in a leaf handler when the mutator splices it into the cmd field at offset 4 of a message (the size dword at offset 0 is rewritten by the harness, so cmd is the only dispatch-relevant header field). The bare global types are emitted too, as a 4-byte splice for messages the fuzzer builds from scratch. The parser is intentionally narrow: it only matches the two SOF_*_TYPE() macros, which keeps the script free of preprocessor logic while covering the full IPC3 dispatch surface (topology, PM, component, DAI, stream, trace, probe, debug and test groups). The cmd-group-to-global-type mapping is an explicit table so a renamed or newly added group is an obvious one-line edit. A missing header is a hard error. Usage: python3 scripts/gen_fuzz_ipc3_dict.py -o <output.dict> Coverage impact, measured on the coverage-instrumented harness (IPC3, empty corpus, 5 seeds x 60 s, paired dict-minus-nodict means): libFuzzer edges (cov) +7 libFuzzer features (ft) +266 (positive on every seed) IPC subsystem lines +9 (+0.4 pp of src/ipc) IPC3's dispatch surface is small enough that line coverage saturates either way at 60 s; the dictionary's consistent benefit is in feature (edge x counter) combinations. The change is primarily about completeness and maintainability, replacing opaque hand-tuned tokens with a documented, header-synchronised superset (64 entries). The output is not committed; it is regenerated from the header on demand (for example by CI into a temporary file) and deployments may extend it with site-specific tokens. Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
Add a -d <dfile> option to fuzz.sh that forwards -dict=<dfile> to the libFuzzer executable. A dictionary of known IPC command dwords and enum constants helps the mutator reach deeper dispatch paths that random mutation rarely hits. The option is deliberately non-fatal: a missing or empty dictionary file only emits a warning and the fuzzer runs without it. The dictionary is a pure enhancement and must never block a fuzzing run, for example when its generator fails after an unrelated header change in CI. fuzz.sh stays agnostic of how the dictionary is produced. The caller is responsible for generating and passing the file. Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
Generate the IPC3 / IPC4 libFuzzer dictionary on the fly from the in-tree headers and pass it to fuzz.sh through the new -d option, so the CI fuzzing job benefits from dictionary-guided mutation without committing a generated file to the tree. The dictionary is written to $RUNNER_TEMP, which is provisioned and cleaned up by the runner and lives outside the checkout, so nothing is committed. The generation step is best-effort (continue-on-error): if the generator ever breaks, the job still fuzzes without the dictionary rather than failing, and fuzz.sh -d tolerates the missing file. Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
kv2019i
approved these changes
Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds header-synchronised libFuzzer dictionary generators for the IPC3 and IPC4 fuzz targets and wires them into CI.
Motivation:
Most random 4-byte prefixes are rejected by the IPC type-field switch, so the fuzzer reached the deeper dispatch handlers only rarely. The dictionaries seed the mutator with real command dwords and enum constants so those paths are hit far sooner.
What's added:
Two small, dependency-free Python generators (scripts/gen_fuzz_ipc{3,4}_dict.py) that harvest the dispatch constants directly from the in-tree headers (so they never drift out of sync), an optional -d passthrough in scripts/fuzz.sh, and an ipc_fuzzer.yml step that regenerates the dictionary into a temporary file per run and passes it to the fuzzer. The dictionary is intentionally not committed (it is regenerated on demand) and the whole feature is non-fatal: if generation ever fails (e.g. after a header rename) the job simply fuzzes without it, so the existing flow is never broken.