SweetSyntax π A generic parser and AST explorer
for analyzing programming languages
nimble install sweetsyntax
- Fast, compiled and efficient (check benchmarks section)
- Generic parser & AST explorer
- Embeddable in other languages via FFI π Lua, JavaScript (N-API), Ruby, Python, PHP
- Easy-to-use API for integration into various applications
- Built-in syntax support for: C, Crystal, D lang, Go, JavaScript, Nim, PHP, Python, Ruby, and Rust
- Zero-copy parsing using MemFiles
- Context-aware error reporting while parsing
- Written in Nim
SweetSyntax is a powerful and flexible generic parser and AST explorer for analyzing programming languages! It is designed to be integrated into other applications, such as code editors, documentation generators, linting tools and other sweet things!
Parse any language by defining its grammar in a YAML specification file: tokens, operators (prefix, infix, postfix, assignment), statement keywords, block delimiters, and feature flags (arrow functions, generators, async/await, template literals). The parser uses a Pratt parsing approach with a language-agnostic core and per-language statement handlers.
Key capabilities:
- Lexer: Generic tokenizer handling identifiers, literals (int, float, hex, octal, binary, bigint, string, regex), comments, and operators
- Parser: Pratt (precedence-climbing) parser with configurable operator precedence and associativity
- AST: Typed node tree with support for statements, expressions, infix/prefix/postfix operations, function declarations, and more
- YAML-driven: Language syntaxes are pure YAML, allowing for custom statement handlers
SweetSyntax is written in Nim, and thanks to Nim's versatile compilation model, can be embedded natively into a wide range of host languages. This is WIP via https://github.com/openpeeps/clue toolkit
| Language | Integration |
|---|---|
| Lua | Load the compiled .so/.dll via LuaJIT FFI or a lightweight C binding |
| JavaScript | Use as a Node.js native addon via N-API |
| Ruby | Bundle as a Ruby C extension |
| Python | Call through Python's CFFI or ctypes |
| PHP | Expose as a PHP extension written in C |
The Nim library compiles to a small, self-contained shared object that any FFI-capable language can load, making SweetSyntax a portable parsing engine for your polyglot projects.
import sweetsyntax
# Parse a file β the library detects the language from the extension
let program = parseJavaScript("app.js")
echo toJson(program)let program = parseScript("module.ts", jsHandlers,
features = {featAsync, featArrowFn, featGenerators,
featLabeledStmt, featTemplateLit})proc myHandlers(p: var GenericParser) =
stmtHandler p, "myKeyword":
walk p
result = Node(kind: nkStatement,
children: @[Node(kind: nkIdent, name: "myKeyword"), parseExpression(p)])
let program = parseScript("file.ext", myHandlers,
features = {featAsync, featArrowFn})SweetSyntax has built-in support for context-aware reporting. For example:
return a == null || b == null ## NaN : a < b ? -1 : a > b ? 1 : a >= b ? 0 : NaN;
^
Error (2:33) Unexpected prefix token: '#'
SweetSyntax is built for speed. Below is a hyperfine benchmark parsing and validating a full copy of d3.js (v7.9.0, ~20k lines, unminified, from cdnjs.com). The entire pipeline (lexing, parsing, and AST generation) completes in under 120ms on my π₯ rastafarian Ryzen 5 with 6 cores/12 threads:
hyperfine --runs 4 './sweetsyntax_benc_d3'
Benchmark 1: ./sweetsyntax_benc_d3
Time (mean Β± Ο): 116.5 ms Β± 1.1 ms [User: 107.5 ms, System: 7.1 ms]
Range (min β¦ max): 115.9 ms β¦ 118.2 ms 4 runs
- π Found a bug? Create a new Issue
- π Wanna help? Fork it!
 |
Switch to Open-Source LLMs via OpenCode GO, choosing from a variety of powerful models such as DeepSeek, Qwen, Kimi, GLM-5, MiniMax, MiMo. π Use our referral link to get started! |
LGPLv3 license. Made by Humans from OpenPeeps.
Copyright OpenPeeps & Contributors β All rights reserved.