-
Notifications
You must be signed in to change notification settings - Fork 0
feat: reconcile Forward Email domain settings and aliases from git #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
ee59526
feat: reconcile Forward Email domain settings and aliases from git
srgvg 5527548
fix(forward-email): harden writes, validation and drift after review …
srgvg d473257
fix(forward-email): address the ten Codex review threads on #1
srgvg f76b8ee
fix(forward-email): reject unknown keys in the forward_email block an…
srgvg ec06620
fix(forward-email): strict responder schema, per-domain zone dir, exa…
srgvg 2072407
fix(forward-email): guard mailbox updates and no-target aliases, reje…
srgvg eeaf3d7
fix(forward-email): refuse --dry-run with --export, surface plan erro…
srgvg File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,238 @@ | ||
| """Reconcile Forward Email domain settings and aliases with per-domain files in git. | ||
|
|
||
| Opt-in only: acts solely on domains listed under `forward_email:` in config.yaml. | ||
| `--dry-run` (default) reads the account and prints the diff; `--doit` writes it. | ||
| Domains are never created or deleted — a claimed domain missing from the account is | ||
| an error. `--prune` (off by default) deletes aliases absent from git, except mailboxes | ||
| (`has_imap` or stored mail), which block the run instead. | ||
|
|
||
| Modes: | ||
| plan / apply diff desired state against the account (default) | ||
| --export write `<directory>/<domain>.yaml` from live state (bootstrap / re-baseline) | ||
| --drift compare FE's generated DNS records and read-only expectations with the repo | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import os | ||
| import sys | ||
| from pathlib import Path | ||
| from typing import TextIO | ||
|
|
||
| import yaml | ||
|
|
||
| from octodns_gitops.forward_email.api import ForwardEmailApiError, ForwardEmailClient | ||
| from octodns_gitops.forward_email.config import ( | ||
| ForwardEmailConfig, | ||
| ForwardEmailConfigError, | ||
| load_domain_file, | ||
| load_forward_email, | ||
| ) | ||
| from octodns_gitops.forward_email.drift import check_zone | ||
| from octodns_gitops.forward_email.export import export_domain | ||
| from octodns_gitops.forward_email.reconcile import DomainPlan, plan_domain | ||
|
|
||
|
|
||
| def zone_directory(config_path: str) -> Path | None: | ||
| """The YamlProvider's `directory`, resolved relative to config.yaml.""" | ||
| cfg_file = Path(config_path) | ||
| with open(cfg_file) as f: | ||
| cfg = yaml.safe_load(f) or {} | ||
| for prov in (cfg.get("providers") or {}).values(): | ||
| if isinstance(prov, dict) and str(prov.get("class", "")).endswith("YamlProvider"): | ||
| d = Path(prov.get("directory", "./zones")) | ||
| return d if d.is_absolute() else cfg_file.resolve().parent / d | ||
|
srgvg marked this conversation as resolved.
Outdated
|
||
| return None | ||
|
|
||
|
|
||
| def _print_plan(plan: DomainPlan, out: TextIO) -> None: | ||
| n = len(plan.settings) + len(plan.aliases) | ||
| head = f"{n} change(s)" if n else "no changes" | ||
| out.write(f"{plan.domain:<28} {head}\n") | ||
| if plan.settings: | ||
| parts = [f"{c.field} {c.live!r} -> {c.desired!r}" for c in plan.settings] | ||
| extra = ", ".join(f"{k}={v!r}" for k, v in plan.write_only.items()) | ||
| out.write(f" settings: {'; '.join(parts)}") | ||
| out.write(f" (+ write-only, unverifiable: {extra})\n" if extra else "\n") | ||
| for chg in plan.aliases: | ||
| detail = "" if chg.action == "create" else f" [{', '.join(chg.changes)}]" | ||
| out.write(f" alias {chg.action:<6} {chg.name}{detail}\n") | ||
| if plan.unmanaged: | ||
| out.write(f" unmanaged (not in git, not pruned): {', '.join(plan.unmanaged)}\n") | ||
| out.writelines( | ||
| f" expect: {m.field} is {m.live!r}, expected {m.desired!r} (read-only, not written)\n" | ||
| for m in plan.expect_mismatch | ||
| ) | ||
| out.writelines(f" ERROR: {e}\n" for e in plan.errors) | ||
|
|
||
|
|
||
| def _apply(plan: DomainPlan, client, out: TextIO) -> bool: | ||
| """Perform the writes for one domain. Returns False if the prune guard aborted.""" | ||
| body = plan.settings_body() | ||
| if body: | ||
| client.update_domain(plan.domain, body) | ||
| out.write(f" applied settings: {sorted(body)}\n") | ||
| deletes = [c for c in plan.aliases if c.action == "delete"] | ||
| for chg in plan.aliases: | ||
| if chg.action == "create": | ||
| client.create_alias(plan.domain, chg.body) | ||
|
srgvg marked this conversation as resolved.
Outdated
|
||
| elif chg.action == "update": | ||
| client.update_alias(plan.domain, chg.alias_id, chg.body) | ||
| else: | ||
| continue | ||
| out.write(f" applied alias {chg.action} {chg.name}\n") | ||
| if not deletes: | ||
| return True | ||
| # A0: never prune from a single listing — re-fetch and require the same id set, and for | ||
| # every planned delete the same name and still no mailbox (a mailbox may appear under an | ||
| # unchanged id between the two listings). | ||
| before = plan.live_alias_ids | ||
| relisted = {a.get("id"): a for a in client.list_aliases(plan.domain)} | ||
| if before != set(relisted): | ||
| out.write( | ||
| f" ERROR: alias list for {plan.domain} changed between listings " | ||
| f"({len(before)} -> {len(relisted)} ids); prune aborted, re-run\n" | ||
| ) | ||
| return False | ||
| for chg in deletes: | ||
| now = relisted[chg.alias_id] | ||
| if now.get("name") != chg.name or now.get("has_imap") or (now.get("storage_used") or 0) > 0: | ||
| out.write( | ||
| f" ERROR: alias {chg.name} ({chg.alias_id}) changed between listings " | ||
| f"(now name={now.get('name')!r}, has_imap={now.get('has_imap')}, " | ||
| f"storage_used={now.get('storage_used')}); prune aborted, re-run\n" | ||
| ) | ||
| return False | ||
| for chg in deletes: | ||
| client.delete_alias(plan.domain, chg.alias_id) | ||
| out.write(f" applied alias delete {chg.name}\n") | ||
| return True | ||
|
|
||
|
|
||
| def run( | ||
| cfg: ForwardEmailConfig, | ||
| client, | ||
| *, | ||
| domains: list[str] | None, | ||
| doit: bool, | ||
| prune: bool, | ||
| mode: str, | ||
| zone_dir: Path | None, | ||
| out: TextIO, | ||
| ) -> int: | ||
| scope = list(cfg.domains) | ||
| if domains: | ||
| unclaimed = [d for d in domains if d not in cfg.domains] | ||
| if unclaimed: | ||
| out.write(f"not claimed under forward_email.domains: {', '.join(unclaimed)}\n") | ||
| return 2 | ||
| scope = [d for d in scope if d in domains] | ||
|
|
||
| if mode == "plan": | ||
| out.write("APPLYING changes (--doit)\n" if doit else "DRY-RUN (default): no changes\n") | ||
| try: | ||
| live_domains = {d["name"].lower(): d for d in client.list_domains()} | ||
| except Exception as e: # noqa: BLE001 - any failure here means nothing can be judged | ||
| out.write(f"ERROR listing domains: {type(e).__name__}: {e}\n") | ||
| return 1 | ||
|
|
||
| rc = 0 | ||
| for domain in scope: | ||
| if domain not in live_domains: | ||
| out.write(f"{domain:<28} ERROR not in the account (domains are never created from git)\n") | ||
| rc = 1 | ||
| continue | ||
| try: | ||
| live = client.get_domain(domain) | ||
| if mode == "export": | ||
| aliases = client.list_aliases(domain) | ||
| cfg.directory.mkdir(parents=True, exist_ok=True) | ||
| path = cfg.directory / f"{domain}.yaml" | ||
| path.write_text(export_domain(cfg, live, aliases)) | ||
|
srgvg marked this conversation as resolved.
Outdated
|
||
| out.write(f"{domain:<28} wrote {path} ({len(aliases)} aliases)\n") | ||
| continue | ||
|
|
||
| path = cfg.directory / f"{domain}.yaml" | ||
| desired = load_domain_file(path, domain) | ||
|
|
||
| if mode == "drift": | ||
| findings = [] | ||
| plan = plan_domain(desired, cfg, live, [], prune=False) | ||
| findings += [(m.field, f"{m.field} is {m.live!r}, expected {m.desired!r}") for m in plan.expect_mismatch] | ||
|
srgvg marked this conversation as resolved.
|
||
| zone_file = (zone_dir / f"{domain}.yaml") if zone_dir else None | ||
| if zone_file is None or not zone_file.exists(): | ||
| out.write(f"{domain:<28} no zone file in this repo; DNS records not checked\n") | ||
| else: | ||
| with open(zone_file) as f: | ||
| zone = yaml.safe_load(f) or {} | ||
| settings = {**cfg.settings, **desired.settings} | ||
| for fnd in check_zone(live, zone, expect_mx=not settings.get("ignore_mx_check")): | ||
| findings.append((fnd.field, fnd.message)) | ||
| if findings: | ||
| rc = 1 | ||
| out.write(f"{domain:<28} {len(findings)} finding(s)\n") | ||
| out.writelines(f" {field}: {msg}\n" for field, msg in findings) | ||
| elif zone_file is not None and zone_file.exists(): | ||
| out.write(f"{domain:<28} clean\n") | ||
| continue | ||
|
|
||
| aliases = client.list_aliases(domain) | ||
| plan = plan_domain(desired, cfg, live, aliases, prune=prune) | ||
| plan.live_alias_ids = {a.get("id") for a in aliases} | ||
| _print_plan(plan, out) | ||
| if plan.errors: | ||
| rc = 1 | ||
| continue | ||
| if doit and not plan.is_empty() and not _apply(plan, client, out): | ||
| rc = 1 | ||
| except (ForwardEmailConfigError, ForwardEmailApiError) as e: | ||
| out.write(f"{domain:<28} ERROR {e}\n") | ||
| rc = 1 | ||
| except Exception as e: # noqa: BLE001 - one domain's failure must not abort the others | ||
| out.write(f"{domain:<28} ERROR {type(e).__name__}: {e}\n") | ||
| rc = 1 | ||
| return rc | ||
|
|
||
|
|
||
| def main() -> int: | ||
| p = argparse.ArgumentParser(description="Reconcile Forward Email settings and aliases (opt-in)") | ||
| p.add_argument("--config", default="config.yaml") | ||
| p.add_argument("--domain", action="append", help="Limit to this domain (repeatable)") | ||
| p.add_argument("--doit", action="store_true", help="Perform writes (default dry-run)") | ||
| p.add_argument("--dry-run", action="store_true", help="Preview only (the default; accepted for symmetry)") | ||
| p.add_argument("--prune", action="store_true", help="Delete aliases absent from git (never mailboxes)") | ||
| g = p.add_mutually_exclusive_group() | ||
| g.add_argument("--export", action="store_true", help="Write per-domain files from live state") | ||
| g.add_argument("--drift", action="store_true", help="Check FE DNS records and expectations against the repo") | ||
| args = p.parse_args() | ||
|
|
||
| try: | ||
| cfg = load_forward_email(args.config) | ||
| except ForwardEmailConfigError as e: | ||
| print(f"config error: {e}", file=sys.stderr) | ||
| return 2 | ||
| if cfg is None: | ||
| print("Forward Email not configured (no forward_email: block in config.yaml).") | ||
| return 0 | ||
| token = os.environ.get(cfg.token_env) | ||
| if not token: | ||
| print(f"missing env var {cfg.token_env} (forward_email.token)", file=sys.stderr) | ||
| return 2 | ||
|
|
||
| mode = "export" if args.export else "drift" if args.drift else "plan" | ||
| client = ForwardEmailClient(token) | ||
| return run( | ||
| cfg, | ||
| client, | ||
| domains=[d.lower().rstrip(".") for d in args.domain] if args.domain else None, | ||
| doit=args.doit and mode == "plan", | ||
|
srgvg marked this conversation as resolved.
|
||
| prune=args.prune, | ||
| mode=mode, | ||
| zone_dir=zone_directory(args.config) if mode == "drift" else None, | ||
| out=sys.stdout, | ||
| ) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| """Forward Email account settings as GitOps: opt-in per repo, per-domain desired state.""" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.