diff --git a/assets/pfsense-rules-wan.png b/assets/pfsense-rules-wan.png index 1807943..bfbc8fd 100644 Binary files a/assets/pfsense-rules-wan.png and b/assets/pfsense-rules-wan.png differ diff --git a/docs/architecture-overview.md b/docs/architecture-overview.md index fdbdd46..f75dde6 100644 --- a/docs/architecture-overview.md +++ b/docs/architecture-overview.md @@ -1,5 +1,5 @@ - @@ -31,7 +31,7 @@ The actual answer here is **four boundaries, each doing a different job**: | 🔴 WAN → edge | Anyone on the internet | Public reverse proxy, firewall-filtered | The only boundary facing genuinely untrusted traffic — gets the strictest, most isolated treatment | | 🔵 Browsers → user-facing proxy | A human, on a device that could be compromised, phished, or simply lost | Mutual TLS (client cert required) + SSO | The highest-value target for credential theft — a stolen password alone isn't enough without the client cert too | | 🟢 App-to-app | A container that's already inside the network, already provisioned by the same pipeline as everything else | IP-allowlist + SSO fallback, no mTLS | Full mTLS here would mean managing certificate lifecycle for every internal service-to-service call — real overhead for traffic that's already constrained to known, provisioned sources. The risk profile doesn't justify the same control as human-operated browsers. | -| 🟡 LAN infra-to-infra | Non-HTTP traffic — metrics, logs, NTP, agent reporting | Explicit per-flow firewall rule, no proxy at all | Can't go through a reverse proxy — it isn't HTTP. The firewall itself is the only layer that *can* enforce this one. | +| 🟡 LAN infra-to-infra | Non-HTTP traffic — metrics, logs, NTP, agent reporting | Explicit per-flow rule at each host's own firewalld — the real gate, since same-subnet traffic never transits a router | Can't go through a reverse proxy — it isn't HTTP. Each host's own firewall is the layer that actually sees this traffic; pfSense carries a mirrored ruleset for anything that does route through it, as defense-in-depth. | Every boundary above only means something if a service can't just skip it by reading the wrong header: **backend services trust proxy headers, never the raw client IP.** If that weren't true, the mTLS/SSO enforcement happening at the proxy layer would be theater — anything downstream could just claim to be a trusted IP and walk straight through. The proxy is the only thing allowed an opinion on who a client really is. @@ -56,11 +56,11 @@ A single-router design fails this test completely — one outage, one misconfigu > > The second fix was conceptual, and the bigger one: for a long time the app-to-app boundary simply didn't exist. Internal service traffic crossed a flat, unauthenticated mesh network — anything on it could reach anything else on it, no proxy, no identity check, nothing. That's exactly the "you're on the network, you're trusted" pattern this document opens by calling the easy, wrong answer. Replacing it with the two-zone model in [`docs/dns-naming.md`](dns-naming.md) is what gave app-to-app traffic a real boundary for the first time. > -> The third fix found the same flaw hiding somewhere less obvious: plain LAN-to-LAN traffic — metrics, logs, NTP, agent reporting — still had a blanket allow at the firewall, even after the app-to-app zone had its own identity-checked proxy. Any host could reach any other host; the only thing standing between "one compromised container" and "everything else on the LAN" was whatever that individual host's own firewall happened to be configured to do, which is a coin flip, not a boundary. Closing it meant giving up "it just works" for LAN infra traffic and writing an explicit rule for every flow that's actually supposed to exist — the same trade this document makes everywhere else: a boundary only counts if crossing it takes a decision, not an accident of network topology. +> The third fix found the same flaw hiding somewhere less obvious: plain LAN-to-LAN traffic — metrics, logs, NTP, agent reporting — still had a blanket allow, even after the app-to-app zone had its own identity-checked proxy. Any host could reach any other host; the only thing standing between "one compromised container" and "everything else on the LAN" was whatever that individual host's own firewall happened to be configured to do, which is a coin flip, not a boundary. Closing it meant writing an explicit per-flow rule everywhere that boundary is actually enforced: each host's own firewalld (the real gate — this LAN is one flat /24, so traffic between two hosts on it never transits a router at all) and, for anything that does route through pfSense itself, a matching explicit-allow-plus-catch-all-deny ruleset there too, as defense-in-depth and a stated record of intent. > > Three different places, the same failure mode, the same fix each time — that's not a coincidence, it's what happens when you actually go looking for where "trusted by default" is still hiding. -That third fix isn't just a claim — it's the actual live ruleset: +That third fix isn't just a claim. Here's pfSense's half of it — even though genuine host-to-host traffic on this flat LAN never actually reaches it: ![pfSense LAN catch-all deny rule: block+log east-west traffic not explicitly allowed](../assets/pfsense-rules-lan-catchall-deny.png) @@ -69,9 +69,9 @@ That third fix isn't just a claim — it's the actual live ruleset:
See the rest of the actual pfSense configuration -**WAN — only three things pass inbound, everything else default-deny:** +**WAN — only four things pass inbound, everything else default-deny:** SSH management, HTTPS via HAProxy, an HTTP→HTTPS redirect, and a STUN port-forward for the NetBird mesh VPN's NAT traversal. -![pfSense WAN rules: SSH management, HTTPS via HAProxy, HTTP redirect, bogon block](../assets/pfsense-rules-wan.png) +![pfSense WAN rules: SSH management, HTTPS via HAProxy, HTTP redirect, NetBird STUN, bogon block](../assets/pfsense-rules-wan.png) **LAN — fleet observability and security-agent reporting, one explicit rule per flow:** @@ -95,16 +95,16 @@ That third fix isn't just a claim — it's the actual live ruleset: ## The Other Six Traffic Laws -Each of these gets its own full mechanism doc elsewhere in this repo — this table is the map, not the territory: +Laws 1–2 are covered above (the trust-boundary table); these are 3–8. Each of these gets its own full mechanism doc elsewhere in this repo — this table is the map, not the territory: | # | Law | Failure behavior | |---|---|---| -| 3. Secrets | Applications never talk to Vault directly — one hop through a local agent only, with one narrow exception: a dedicated SSH-certificate refresh reuses that agent's token to call Vault directly. See [`docs/secrets-and-pki.md`](secrets-and-pki.md) and [`docs/secret-distribution.md`](secret-distribution.md). | Vault down → secret/certificate renewal stops, services run until their current token or cert expires — no immediate disruption. The one flow that doesn't degrade this gently: the SSH-certificate exception above has no static-key fallback, so a sustained outage eventually locks the automation identity out of the fleet entirely, not just "stale but running." | -| 4. DNS | Which internal zone a hostname resolves in determines which trust boundary (above) a request enters. See [`docs/dns-naming.md`](dns-naming.md). | DNS down → app-to-app resolution fails; a direct-IP fallback exists, and it bypasses the SSO gate — but only for traffic already coming from an IP on the trusted allowlist. Anything else still hits Authentik regardless of DNS. | -| 5. Observability | Logs and metrics push asynchronously — never inline with application traffic, full stop. See [`docs/observability.md`](observability.md) and [`docs/security-monitoring.md`](security-monitoring.md). | Monitoring down → collection stops; every application service keeps running, completely unaffected. | -| 6. Notifications | Two independent branches (message-queue and HTTP webhook) converge on one formatter; neither is part of any service's runtime dependency chain. See [`docs/notifications.md`](notifications.md). | Notification delivery is event-driven, not persistent inline execution — a failure here doesn't touch application traffic either way. | -| 7. CI/CD | Build/deploy traffic never sits inline with runtime; schedule-triggered, not push-triggered. See [`docs/registry-and-build.md`](registry-and-build.md). | A failed deploy doesn't affect already-running services — enforced by the redeploy step's own rollback logic. | -| 8. Management | Administrative access is its own separate plane — one dedicated identity, non-standard ports, explicit per-host firewall rules from one source only on the LAN, plus a separate zero-trust mesh giving that same access a scoped path in from off-LAN too. See [`docs/netbird-zero-trust.md`](netbird-zero-trust.md). | On-LAN admin access never depended on the mesh and is unaffected if it goes down; off-LAN access specifically loses its only path until the mesh recovers. Every running service is unaffected either way. | +| 3 | **Secrets.** Applications never talk to Vault directly — one hop through a local agent only, with one narrow exception: a dedicated SSH-certificate refresh reuses that agent's token to call Vault directly. See [`docs/secrets-and-pki.md`](secrets-and-pki.md) and [`docs/secret-distribution.md`](secret-distribution.md). | Vault down → secret/certificate renewal stops, services run until their current token or cert expires — no immediate disruption. The one flow that doesn't degrade this gently: the SSH-certificate exception above has no static-key fallback, so a sustained outage eventually locks the automation identity out of the fleet entirely, not just "stale but running." | +| 4 | **DNS.** Which internal zone a hostname resolves in determines which trust boundary (above) a request enters. See [`docs/dns-naming.md`](dns-naming.md). | DNS down → app-to-app resolution fails; a direct-IP fallback exists, and it bypasses the SSO gate — but only for traffic already coming from an IP on the trusted allowlist. Anything else still hits Authentik regardless of DNS. | +| 5 | **Observability.** Logs and metrics push asynchronously — never inline with application traffic, full stop. See [`docs/observability.md`](observability.md) and [`docs/security-monitoring.md`](security-monitoring.md). | Monitoring down → collection stops; every application service keeps running, completely unaffected. | +| 6 | **Notifications.** Two independent branches (message-queue and HTTP webhook) converge on one formatter; neither is part of any service's runtime dependency chain. See [`docs/notifications.md`](notifications.md). | Notification delivery is event-driven, not persistent inline execution — a failure here doesn't touch application traffic either way. | +| 7 | **CI/CD.** Build/deploy traffic never sits inline with runtime; schedule-triggered, not push-triggered. See [`docs/registry-and-build.md`](registry-and-build.md). | A failed deploy doesn't affect already-running services — enforced by the redeploy step's own rollback logic. | +| 8 | **Management.** Administrative access is its own separate plane — one dedicated identity, non-standard ports, explicit per-host firewall rules from one source only on the LAN, plus a separate zero-trust mesh giving that same access a scoped path in from off-LAN too. See [`docs/netbird-zero-trust.md`](netbird-zero-trust.md). | On-LAN admin access never depended on the mesh and is unaffected if it goes down; off-LAN access specifically loses its only path until the mesh recovers. Every running service is unaffected either way. | The pattern repeats across all eight laws, not just the trust-boundary story above: **control-plane failure never takes down the data plane.** Every "failure behavior" row above ends the same way — the thing that broke stops working, and everything else keeps running regardless. That's not a coincidence; it's the same design principle applied consistently everywhere, not just to network trust.