Skip to content

Replace libdparse with dmd as a library in static code analysis - #971

Open
Albert24GG wants to merge 185 commits into
dlang-community:masterfrom
Albert24GG:libdmd
Open

Albert24GG wants to merge 185 commits into
dlang-community:masterfrom
Albert24GG:libdmd

Conversation

@Albert24GG

Copy link
Copy Markdown

This is just a follow-up of #964 to hopefully get it merged.

RazvanN7 and others added 30 commits August 29, 2025 13:16
…n with dmd (dlang-community#17)

* replace libdparse in objectconst functionality + unittests integration with dmd

* updated dmd

* run tests

* use templates

* visit aggregate declaration

* updated dmd

* solve linter seg fault

* get rid of dup + refactor

* fix typo
* replace libdparse in delete check

* delete comment
* Update README

* Add dmd-as-a-library submodule (dlang-community#2)

* Add GH Actions build script (dlang-community#4)

* Removed libdparse from imports print functionality (dlang-community#3)

* Fix failing tester + add unittest for imports (dlang-community#7)

* Add style checker (dlang-community#9)

* Update action to build dlang fork

* Fix linter errors

* Add dmd dependencies to dub.json

* Add dmd dependencies to build.bat

* Replace libdparse in enum array functionality

* replace libdparse in objectconst functionality + unittests integration with dmd (dlang-community#17)

* replace libdparse in objectconst functionality + unittests integration with dmd

* updated dmd

* run tests

* use templates

* visit aggregate declaration

* updated dmd

* solve linter seg fault

* get rid of dup + refactor

* fix typo

* update dmd to latest version

Co-authored-by: RazvanN7 <razvan.nitu1305@gmail.com>
Co-authored-by: Eduard Staniloiu <edi33416@gmail.com>
* update dmd

* update dmd
* replace libdparse in imports sortedness visitor

* minor refactor
gedaiu added 27 commits October 26, 2026 09:51
Restores the original test expectations for the warning ranges: missing
parameters are reported on the parameter name, a missing Returns section
on the return type, and missing Throws sections and documented
parameters that do not exist on the function name. The checks pass the
declaration around instead of a line and column so each warning can
find its own range.
nameRange now uses it, and checks that only know where a declaration
starts can find the name they report.
Restores the original test expectations for the warning range.
Restores the original test expectations for the warning range.
Restores the original test expectations for the warning range.
Restores the original test expectations for the warning range.
Restores the original test expectations for the warning range.
Restores the original test expectations for the warning range.
Restores the original test expectations for the warning range.
Restores the original test case that checks the warning range.
The dmd port dropped the warning for enforce() calls without a message
when std.exception is imported in an enclosing scope. The original tests
are restored unchanged.
The dmd port dropped two parts of the check: the warning for a struct
constructor whose only parameter has a default value, which can never
be used, and the supplemental diagnostics pointing at the two class
constructors that conflict. Both are back, with the original test case
for the struct and a new test for the supplemental diagnostics.
BaseAnalyzerDmd gets an addErrorMessage overload taking supplemental
diagnostics.
dmd no longer has a delete keyword: `delete a;` parses as a variable
declaration and `delete a[1];` is a syntax error, so the DeleteExp
visitor never ran and the test had been commented out. The check now
looks for `delete` at the start of a statement in the source tokens and
keeps the original range and destroy() autofix. The tests are enabled
again; the warning test uses forms dmd can parse, and the autofix test
is the original one.
Restores the original test for identical branches, which underlines the
whole if statement, and reports duplicated operands on the expression.
The other original test only guarded against a libdparse crash on
code dmd rejects, so it is not restored.
The dmd port dereferenced the caught type, which is null for a catch
without an exception specification, and segfaulted. Such a catch
catches everything, so it is reported on the catch keyword again, as the
restored original test expects. Types that are not plain identifiers
are skipped.
Restores the original test case and its warning range.
The dmd port removed the float operator, duplicate attribute, comma
expression and redundant if checks. The options stay so existing
configuration files keep working, and their descriptions now say they
do nothing. dmd itself reports the first three; the redundant if check
has no replacement yet.
They insert the attribute after the parameter list and are named
"Mark function `const`" and so on again, so the expected resolved
autofixes in the IDE integration test are the original ones.
The dmd port never filled in lineOfCodeCount, so reports always said 0.
It is counted from the source tokens again, as --sloc does.

The expected IDE report is the original one again (warning ranges,
autofix names and line count) except for two differences that follow
from dmd: the static if autofix is computed right away instead of being
resolved on request, and statements are counted on the dmd AST.
An alias of a symbol, such as `alias f = (int a) => a;`, has no type,
and the check dereferenced it. This crashed on 14 Phobos modules with
the check enabled.
The dmd port gave every version or static if declaration its own scope,
so `version (Posix) { int b; } int b;` was no longer reported. As
before the port, only the branches of an if/else are treated as
alternatives; a lone branch shares the outer scope.
builtin_property_names_check also reports aliases and templates, and
opequals_tohash_check and has_public_example also see declarations
inside attribute blocks such as @safe { }. These tests pin that down.
dmd adds literals without a source location, such as the 1 of `i++`.
Reading their offset indexed dmd's file table with -1, which threw in
debug builds and crashed release builds on many Phobos modules. Such
expressions are now skipped, and `i++` ends at its operator.
The dmd port limited scopes to a depth of 20 and indexed past the end
of its array at that depth, throwing on several Phobos and druntime
modules. dmd nests deeper than libdparse did, since an if statement and
its block are separate scopes, so the limit is removed as it was before
the port.
dmd gives a local `@trusted struct S {}` no location of its own, and
reading its offset threw. The @trusted attribute is now searched from
the first member instead.
The check looked for the final keyword on the line of the declaration
and threw when it was on another line or came from a final: label, as
in several dmd source files. It now looks among the declaration's own
leading tokens and, as before the dmd port, does not report a final
that comes from an enclosing final: or final { } block. The file is no
longer lexed in the constructor.
@gedaiu

gedaiu commented Oct 26, 2026

Copy link
Copy Markdown

I pushed the work on warning ranges to https://github.com/gedaiu/D-Scanner/tree/dmd-migration. It sits on top of this branch merged with current master, one commit per check, so it should be easy to review or cherry-pick.

@WebFreak001, about the ranges: assertAnalyzerWarningsDMD understands the /+ ^^^ [warn]: ... +/ markers again, and all of the original range expectations are back (229 markers in 43 files). Every check now reports the same start and end columns as the libdparse version. Where dmd could parse the original test code, the tests are the original ones unchanged. In the few places it can't, I changed only what dmd rejects and said so in the commit message. Since dmd records only where a node starts, the ranges are recovered by re-lexing the source with a small set of helpers in source_range.d.

Comparing the branch with master turned up some behaviour the port had lost, and I restored it. That includes the enforce warning in assert_without_msg, the struct constructor warning and the supplemental notes in constructor_check, delete_check (dmd no longer parses delete, so it now works on tokens), the line count in --report, the original @property const autofixes, and the libdparse behaviour of imports_sortedness and label_var_same_name_check. Running every check over Phobos, druntime and dmd's own sources with debug and release builds also found crashes on a catch without a type, symbol aliases, deep nesting, a local @trusted struct and final written on its own line. Those are fixed too, each with a test. With all checks enabled, the 1091 files now go through without a crash or exception, and dub test, make test, it.sh and the self lint all pass locally with dmd and ldc.

A few differences remain, and I'd like your opinion on them. The static if autofix is computed immediately instead of being marked resolvable, since the dmd analyzers have no resolve step yet. Statements in --report are counted on the dmd AST, so the numbers differ from libdparse. Some checks now also see declarations inside attribute blocks such as @safe { }. The float operator, duplicate attribute and comma expression checks are gone because dmd reports those itself, but redundant_if_check has no replacement yet. I left its option in place with a note and would be happy to port it if you want it back.

@Albert24GG, if a pull request against your branch is easier for you than pulling these commits, just say so and I'll open one.

@Albert24GG

Copy link
Copy Markdown
Author

@gedaiu Thanks for the update! I just pulled your changes into my branch. I see there are quite a lot of commits so I don't think I'll have time to look over all the changes myself, but I'll keep the PR up to date with any new progress.

@Albert24GG

Copy link
Copy Markdown
Author

The pipeline probably fails because setup-dlang is outdated, and should be updated to @v2. There is an open PR for that (#967), but it looks stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants