Default deny network policies in all namespaces (default for new clusters)

[weatherhog]

Goal

Harden workload clusters by default: default-deny network policies in all namespaces — not only kube-system/giantswarm — enabled by default for new clusters, with a simple per-namespace opt-out. Existing clusters keep their current behavior on upgrade and can adopt incrementally.

Origin: giantswarm/giantswarm#36605 (customer-request), accepted by Cabbage per discussion there. History: giantswarm/giantswarm#27648 (Shield's opt-in Kyverno generate policy, to be superseded). Companion: giantswarm/giantswarm#32658 (CNP guidelines).

Approach

• Extend network-policies-app (already our default-deny home, installed on every CAPI WC) with:
  • One CiliumClusterwideNetworkPolicy default-deny-all-namespaces selecting all endpoints except namespaces labeled policy.giantswarm.io/default-deny: "false" and except kube-system/giantswarm (which keep today's per-namespace policies).
  • A clusterwide allow-dns CNP as the only default allow. Everything else (in-namespace traffic, kube-apiserver egress, webhook ingress, ingress-controller, metrics scraping) is provided as documented copy-paste policies.
  • A one-shot grandfathering Job that labels all pre-existing non-GS namespaces as opted-out on first enablement — this is what makes "default for new clusters, unchanged for existing clusters" work without the chart knowing cluster age.
• New cluster-chart value global.connectivity.network.defaultNetworkPolicies.mode: all-namespaces | gs-namespaces | none, default all-namespaces in a new major cluster-provider release; deprecated allowAllEgress: true maps to none.
• Discovery/adoption story for customers: Hubble (--verdict DROPPED, already on by default), Cilium policyAuditMode (to be exposed in cilium-app), incremental enforcement via the opt-out label, OBI network monitoring + a Grafana policy-drops dashboard.
• Large docs + comms effort: today default-deny is essentially undocumented (the getting-started tutorial even states there is no default deny).

Key risks (details in stories)

• Workload breakage at enforcement — mitigated by grandfathering + comms; expect support questions from GitOps-created namespaces.
• Cluster-wide layer is Cilium-only (no plain NetworkPolicy equivalent for label-based cluster-wide deny).
• Audit mode is cluster-wide (also un-enforces existing GS denies) — time-boxed use only.
• Opt-out label must never land in cilium-app identity-label exclusions.
• BPF policy-map growth from customer allow policies (see giantswarm/giantswarm#32658).

Stories

See sub-issues.