[scttcper]

This optimizes generated parser code for simple character classes.

A lot of grammars have hot loops that look like this:

identifier = [a-zA-Z_$] [a-zA-Z0-9_$]*
number = [0-9]+ ("." [0-9]+)?
whitespace = [ \t\n\r]*

Peggy currently emits a regexp test for each character consumed by those classes. This PR emits direct charCodeAt comparisons when the class is simple enough to do that exactly: no i flag, no unicode mode, only single-code-unit ranges/characters, and at most 32 parts. Classes that take the new path also skip emitting their now-unused peg$rN regexp.

Anything more complicated still uses the existing regexp path. The 32 part cap is there because huge classes, like the unicode category classes in examples/javascript.pegjs, were 2-3x slower as one long chain of comparisons than as a regexp. The crossover was somewhere between 32 and 64 ranges.

As one real-world benchmark, I used Sentry’s search grammar, which is the grammar we use for parsing search bar queries: https://github.com/getsentry/sentry/blob/master/static/app/components/searchSyntax/grammar.pegjs. That grammar has a lot of key/value filters, so longer searches spend plenty of time in these character-class loops.

Input	Before	After	Speedup
200 short search strings, cycling through common filter/query shapes	2.9267 ms	2.3065 ms	21.2%
one 2.7 KB query with 120 mixed free-text, filter, and grouped terms	0.7829 ms	0.5717 ms	27.0%
two copies of the long query joined with AND, about 5.4 KB	1.6433 ms	1.2222 ms	25.6%
the long query plus a parenthesized copy joined with OR, about 5.4 KB	1.6259 ms	1.2319 ms	24.2%

Before is current main, Node 26.11.1, median of 3 runs where each run is the median of 500 parses.

hyperfine agrees end to end, node startup included: 400 rounds of the short strings went from 1.306 s to 1.047 s (-19.8%), and 300 rounds of the long queries went from 1.309 s to 1.046 s (-20.1%).

The generated parser for that grammar is about 3% bigger (123 KB → 127 KB). For the example grammars it's +0.4% (javascript) to +3% (css).

[scttcper]

let me know if you need the benchmark or something else

[hildjj]

Please rebase, add yourself to AUTHORS, and add to the CHANGELOG.

[hildjj]

Also, this could use a few more tests, please. Include more complicated combinations that both should and should not use the new path.

[scttcper]

Rebased, added myself to AUTHORS, and added a CHANGELOG entry.

Added tests for classes that should take the new path (mixed ranges, inverted, escaped ]\^-, control chars, surrogate ranges) and ones that should stay on the regexp (i, u, \p{}, astral), plus behavior tests for the same shapes.

Also, big classes like the unicode categories in the javascript example were 2-3x slower as a comparison chain, so classes over 32 parts now stay on the regexp. Unused peg$rN regexps are no longer emitted either. Updated the description with fresh numbers against current main.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.49%. Comparing base (05c55c0) to head (df5eeed).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #660   +/-   ##
=======================================
  Coverage   99.49%   99.49%           
=======================================
  Files          34       34           
  Lines        4314     4314           
=======================================
  Hits         4292     4292           
  Misses         22       22           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hildjj]

I'm going to finish looking at this in the morning. I had a quick idea though: is there some number of codepoints at which throwing them into a Set would be a win?

[hildjj]

LGTM. Thanks!

[hildjj]

Released in v5.1.1