Skip to content

Add the ability to use local optimization_config#1186

Open
mu-hun wants to merge 3 commits intofix/1180_local_build_from_cachefrom
local_optimization_config
Open

Add the ability to use local optimization_config#1186
mu-hun wants to merge 3 commits intofix/1180_local_build_from_cachefrom
local_optimization_config

Conversation

@mu-hun
Copy link
Copy Markdown
Member

@mu-hun mu-hun commented Apr 3, 2026

Adds support for downloading and reusing the optimization config locally, allowing users to review the optimization results before deploying changes to the server.

This feature should be managed through the build option in the FiltersRegistry, not by FiltersCompiler.

The workflow

  • percent.json is stored in temp/optimization_config/ and read from there when --use-cache is enabled.
  • After downloading optimization_config/percent.json, edit it locally, then use it in the build.

Both --generate-cache and --use-cache flags now include optimization config handling alongside the existing filter source caching.

Note to @adguard/filters-compiler

@adguard/filters-compiler is set to AdguardTeam/FiltersCompiler#local_optimization_config in package.json.

This requires dist/ to be committed to the branch, as GitHub shorthand installation does not trigger a build step.

Comment thread yarn.lock
@mu-hun mu-hun force-pushed the local_optimization_config branch 2 times, most recently from df6f136 to 424e89f Compare April 8, 2026 15:53
@mu-hun
Copy link
Copy Markdown
Member Author

mu-hun commented Apr 8, 2026

Please specify how to perform the test.

I have just aligned with the flags you created (#1181). In the PR description,

Both --generate-cache and --use-cache flags now include optimization config handling alongside the existing filter source caching.

@mu-hun mu-hun marked this pull request as ready for review April 8, 2026 16:34
@mu-hun mu-hun requested a review from Alex-302 April 8, 2026 16:34
@mu-hun mu-hun force-pushed the local_optimization_config branch from e04f297 to 867ab88 Compare April 27, 2026 08:19
@mu-hun
Copy link
Copy Markdown
Member Author

mu-hun commented Apr 27, 2026

Crash fixed

7691c68. The cache dir is wiped with force: true before generate() runs, so re-running on an existing cache no longer errors.

Verification guideline (Draft)

I tested the steps below, but the result had no output. Maybe I wrote the steps incorrectly or missed something.

  1. Run yarn generate-cache — downloads remote config to temp/optimization_config/
  2. Confirm log output: Using local optimization config from: …/temp/optimization_config
  3. In temp/optimization_config/filters/1/stats.json, set one rule hit count at 9999 (e.g. "||banggood.com^$domain=topic.lt": 9999 from 0 in groups[0].rules)
  4. Run yarn build:local
  5. Confirm log output again (proves local config is active)
  6. Grep compiled output for the rule:

If step 6 returns output, the stats cache is wired correctly end-to-end:

grep -rF "||banggood.com^$domain=topic.lt" platforms/
# → prints matching lines with file paths

After restoring the hit count and running yarn build:local again, the result is no output:

# Rule is absent (before change hit count to 9999) — cache working correctly:
grep -rF "||banggood.com^$domain=topic.lt" platforms/
# → no output

@mu-hun mu-hun force-pushed the local_optimization_config branch from 3e2d9bd to 4bf108a Compare May 7, 2026 05:37
mu-hun added 3 commits May 7, 2026 16:51
Adds support for downloading and reusing optimization config locally,
so the optimization result can be checked before changes are deployed
to the server.

- `percent.json` is saved under `temp/optimization_config_cache/` and read from there when `--use-cache` is active.

Both `--generate-cache` and `--use-cache` flags now include optimization
config handling alongside the existing filter source caching.

`@adguard/filters-compiler` is set to
`AdguardTeam/FiltersCompiler#local_optimization_config` in `package.json`.
This requires `dist/` to be committed to the branch, as GitHub shorthand
installation does not trigger a build step.

Also, I add integration tests for `optimizationConfigLocal` namespace.

Verifies `--use-cache`` calls `setPath()` and `--generate-cache`` calls
`generate()` + `setPath()` with the correct cache path.

Dynamically imports build.js per test after `vi.resetModules()` to
re-execute its module-level side effects with controlled `process.argv`.
Mocks `fs`, `@adguard/filters-compiler`, and `find_files` to stay isolated.
Vitest test (optimization-e2e.test.js) compiles filter_1 against
a self-contained tmp fixture with two synthetic rules — hits=0 rule
must be absent from *_optimized.txt, hits=9999 rule must be present.
No network, no prereqs. Runs in CI via existing yarn test.

Local bash script (optimization-integration-e2e.sh) covers the same
contract against the real generate-cache output. Key fixes baked in:
- strips CRLF (\r) from grepped rule text to match compiler trim()
- scopes grep to *_optimized.txt (non-optimized 1.txt retains all rules)
- sets minPercent=0/maxPercent=100 so percent check passes with only
  2 rules in stats vs ~17k in the real filter

Adds yarn test:optimization-e2e script entry and updates AGENTS.md.
@mu-hun mu-hun force-pushed the local_optimization_config branch from 4bf108a to ac373e9 Compare May 7, 2026 09:46
@mu-hun
Copy link
Copy Markdown
Member Author

mu-hun commented May 7, 2026

The workflow should be followed as follows:

  • percent.json is stored in temp/optimization_config/ and read from there when --use-cache is enabled.
  • After downloading optimization_config/percent.json, edit it locally, then use it in the build.

However, optimizationConfigLocal.generate downloads stats files via the downloaded percent.json without the ability to edit them locally, as in: FiltersCompiler/blob/2231ebc5/src/main/optimization.js#L29-L42

This needs to be updated. Also, need to fix Build CI Failed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants