Skip to content

graphify --update: ghost-node pruning silently no-ops when node source_file is relative (deleted_files is absolute) #2012

Description

@4vbm4il-byte

Summary

The --update incremental-merge flow's ghost-node pruning step compares each existing node's source_file against detect_incremental()['deleted_files'] with a plain membership check:

to_remove = [n for n, d in G_existing.nodes(data=True) if d.get('source_file') in deleted]

deleted_files is always a list of absolute filesystem paths (confirmed via graphify.detect.detect_incrementaldeleted_files is built from manifest keys re-anchored to absolute form via load_manifest(manifest_path, root=root), then compared against full["files"], which are also absolute).

However, the extraction subagent prompt template in SKILL.md (the semantic-extraction schema dispatched to Claude subagents) explicitly instructs source_file to be a path relative to the corpus root:

Node ID format: ...
"source_file": "relative/path"

So on any corpus where nodes were extracted per the documented schema, d.get('source_file') is relative and deleted contains only absolute paths. The in check can never match. The pruning step then reports something like:

56 file(s) deleted since last run, but no ghost nodes were present - no drift.

...even when the deleted files' nodes are genuinely still present in the graph as ghosts.

Repro

  1. Run a full /graphify pass on a corpus (nodes get relative source_file, per the documented schema).
  2. Delete a folder of files that were part of that corpus.
  3. Run /graphify --update.
  4. Observe: deleted_files correctly lists the deleted files (absolute paths). The pruning step reports "no ghost nodes... no drift." The nodes for the deleted files are still present in graph.json.

Verified this by manually diffing node IDs between the pre- and post-update graph.json: 42 nodes whose source files had been deleted were still present after --update reported no drift. Re-running the exact same prune logic with paths normalized to relative-to-root before the membership check correctly identified and removed all 42.

Suggested fix

Normalize both sides to the same path form before the membership check in the --update pruning snippet, e.g.:

deleted_rel = {str(Path(f).relative_to(root)) for f in deleted}
to_remove = [n for n, d in G_existing.nodes(data=True) if d.get('source_file') in deleted_rel]

(or, alternatively, standardize on absolute source_file everywhere and update the extraction prompt schema/docs accordingly — either direction works, but the two need to agree.)

Environment

  • graphify (graphifyy package) 0.9.18
  • macOS, Python 3.10
  • Corpus: ~3,600-node Obsidian vault, multiple prior --update runs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions