Summary
The --update incremental-merge flow's ghost-node pruning step compares each existing node's source_file against detect_incremental()['deleted_files'] with a plain membership check:
to_remove = [n for n, d in G_existing.nodes(data=True) if d.get('source_file') in deleted]
deleted_files is always a list of absolute filesystem paths (confirmed via graphify.detect.detect_incremental — deleted_files is built from manifest keys re-anchored to absolute form via load_manifest(manifest_path, root=root), then compared against full["files"], which are also absolute).
However, the extraction subagent prompt template in SKILL.md (the semantic-extraction schema dispatched to Claude subagents) explicitly instructs source_file to be a path relative to the corpus root:
Node ID format: ...
"source_file": "relative/path"
So on any corpus where nodes were extracted per the documented schema, d.get('source_file') is relative and deleted contains only absolute paths. The in check can never match. The pruning step then reports something like:
56 file(s) deleted since last run, but no ghost nodes were present - no drift.
...even when the deleted files' nodes are genuinely still present in the graph as ghosts.
Repro
- Run a full
/graphify pass on a corpus (nodes get relative source_file, per the documented schema).
- Delete a folder of files that were part of that corpus.
- Run
/graphify --update.
- Observe:
deleted_files correctly lists the deleted files (absolute paths). The pruning step reports "no ghost nodes... no drift." The nodes for the deleted files are still present in graph.json.
Verified this by manually diffing node IDs between the pre- and post-update graph.json: 42 nodes whose source files had been deleted were still present after --update reported no drift. Re-running the exact same prune logic with paths normalized to relative-to-root before the membership check correctly identified and removed all 42.
Suggested fix
Normalize both sides to the same path form before the membership check in the --update pruning snippet, e.g.:
deleted_rel = {str(Path(f).relative_to(root)) for f in deleted}
to_remove = [n for n, d in G_existing.nodes(data=True) if d.get('source_file') in deleted_rel]
(or, alternatively, standardize on absolute source_file everywhere and update the extraction prompt schema/docs accordingly — either direction works, but the two need to agree.)
Environment
- graphify (
graphifyy package) 0.9.18
- macOS, Python 3.10
- Corpus: ~3,600-node Obsidian vault, multiple prior
--update runs
Summary
The
--updateincremental-merge flow's ghost-node pruning step compares each existing node'ssource_fileagainstdetect_incremental()['deleted_files']with a plain membership check:deleted_filesis always a list of absolute filesystem paths (confirmed viagraphify.detect.detect_incremental—deleted_filesis built frommanifestkeys re-anchored to absolute form viaload_manifest(manifest_path, root=root), then compared againstfull["files"], which are also absolute).However, the extraction subagent prompt template in
SKILL.md(the semantic-extraction schema dispatched to Claude subagents) explicitly instructssource_fileto be a path relative to the corpus root:So on any corpus where nodes were extracted per the documented schema,
d.get('source_file')is relative anddeletedcontains only absolute paths. Theincheck can never match. The pruning step then reports something like:...even when the deleted files' nodes are genuinely still present in the graph as ghosts.
Repro
/graphifypass on a corpus (nodes get relativesource_file, per the documented schema)./graphify --update.deleted_filescorrectly lists the deleted files (absolute paths). The pruning step reports "no ghost nodes... no drift." The nodes for the deleted files are still present ingraph.json.Verified this by manually diffing node IDs between the pre- and post-update
graph.json: 42 nodes whose source files had been deleted were still present after--updatereported no drift. Re-running the exact same prune logic with paths normalized to relative-to-root before the membership check correctly identified and removed all 42.Suggested fix
Normalize both sides to the same path form before the membership check in the
--updatepruning snippet, e.g.:(or, alternatively, standardize on absolute
source_fileeverywhere and update the extraction prompt schema/docs accordingly — either direction works, but the two need to agree.)Environment
graphifyypackage) 0.9.18--updateruns