🕸️ Zero-Trust Mesh Overlay

Note

A self-hosted NetBird WireGuard mesh gives every fleet host — and every device that needs to reach it — its own stable overlay identity, independent of where that device physically sits. Access is granted per-identity through mesh policy, not inferred from LAN position. The mesh is deny-by-default: nothing crosses it unless an explicit policy allows it, the same instinct behind every other boundary in docs/architecture-overview.md, applied to remote and roaming access specifically.

Why A Mesh, Not a Classic VPN

A traditional VPN concentrator was the obvious alternative, and it was rejected for the same reason a flat LAN got rejected everywhere else in this fleet: it's one more device sitting in a path, and reaching it means punching a hole to a single choke point rather than establishing an identity-scoped connection to exactly the host that's actually needed. A mesh instead gives every peer — fleet host or personal device — its own stable identity (<host>.netbird.selfhosted, a 100.91.0.0/16 overlay address) and lets WireGuard tunnels form directly, peer-to-peer, falling back to a relay only when direct connectivity isn't possible. Policy is written once, per identity or group, and travels with the peer regardless of which network it's physically on.

This is the actual shape of it — the control plane coordinates, but it's deliberately not where the traffic goes:

That's the actual difference from a VPN concentrator, drawn rather than just asserted: a concentrator puts itself in every data path by design; here the control plane's spokes only ever carry coordination traffic, and every thick line is two peers talking directly to each other.

Control Plane

One netbirdio/netbird-server container runs Management, Signal, the embedded relay, and NetBird's own embedded IdP together — combined-server mode, not four separate services. A digest-pinned netbirdio/dashboard container serves the admin SPA. This is the real container list, not a claimed one:

$ docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
NAMES               IMAGE                                               STATUS
netbird-proxy       caddy:2-alpine                                      Up 38 hours
netbird-server      netbirdio/netbird-server:0.77.1                     Up 38 hours
netbird-dashboard   f3eb26c93ca9                                        Up 38 hours

Configuration is Vault-Agent-rendered, never committed — secrets live in Vault under a dedicated AppRole and policy scoped to this one host, the same pattern as every other secret in docs/secrets-and-pki.md. The identity provider is NetBird's own embedded one, not Authentik — Authentik is deliberately LAN-only in this fleet and publishing it wasn't in scope, so there's no OIDC migration path here; the first-login owner account is the permanent identity for this system specifically.

Self-hosted rather than the vendor's own managed cloud offering, for the same reason every other security-critical system in this fleet is self-hosted rather than SaaS: every peer's identity, every access policy, every session record stays inside the same trust boundary Vault protects in docs/secrets-and-pki.md, instead of a third party holding the map of exactly which devices exist and what they're allowed to reach. For a mesh whose entire job is deciding who gets in, that map is itself sensitive — running the control plane anywhere else would mean trusting someone else's infrastructure with the fleet's own access model.

Addressing:

Item	Value
Overlay v4 range	`100.91.0.0/16`
Overlay v6 range	a ULA `/64`
Peer identity	dashboard name = short hostname; mesh DNS label = `<host>.netbird.selfhosted`
Peer WireGuard	port `51820`, kernel interface `wt0`

The Edge Routing Problem, and What Actually Broke

Warning

Learned the hard way during bring-up, not designed in from the start. NetBird's Management and Signal services need native gRPC over HTTP/2-cleartext (h2c) end-to-end — any downgrade to HTTP/1.1 anywhere in the chain kills the bidirectional stream with an unexpected EOF, and the client agent just wedges with no useful error. The edge firewall's own reverse proxy can carry h2c to a single plain-HTTP backend, but it can't perform NetBird's own content-type/path split between the management API, the relay, and the dashboard SPA — so a small in-stack proxy does exactly that split, one hop further in. A second, unrelated thing broke the same week: the public DNS record for this service has to stay DNS-only, not proxied through a CDN — a proxying CDN returns an outright 403 to the gRPC-speaking agent, no explanation given, and the fix was disabling that proxying entirely for this one record rather than trying to work around it.

The real path a request takes:

And this is the actual splitter doing that work — the real config, not a description of the logic:

$ cat Caddyfile
{
	admin off
	auto_https off
	servers {
		protocols h1 h2 h2c
	}
}

:33000 {
	# native gRPC + gRPC-web (any application/grpc* content-type) -> server over h2c
	@grpc header Content-Type application/grpc*
	handle @grpc {
		reverse_proxy h2c://netbird-server:80
	}

	# REST, OAuth, relay/signal WebSockets, and the gRPC service paths -> server
	@srv path /api/* /oauth2/* /relay* /ws-proxy/* /signalexchange* /management*
	handle @srv {
		reverse_proxy netbird-server:80
	}

	# everything else -> dashboard SPA
	handle {
		reverse_proxy netbird-dashboard:80
	}
}

Fleet Enrolment

A dedicated Ansible role installs the mesh agent and enrols a host, opt-in per host via the same tag-driven mechanism as every other role in docs/iac-pipeline.md — nothing joins the mesh by default just because it exists. Enrolment uses a reusable, class-scoped setup key fetched live from the secrets platform at play time, never stored in the repo, and read only on a host's first join. Because peer approval is required, a brand-new peer pauses for an operator to approve it in the dashboard before it gets any mesh access at all — there's no silent auto-join path.

All fleet hosts are enrolled today, plus a small number of personal devices — being on the mesh and being allowed anywhere on it are two separate, independently-granted things, and not every enrolled device gets the same access. Fleet servers run with local DNS resolution disabled on the mesh interface deliberately — they're not roaming, and the fleet's own internal DNS already handles them.

Split-DNS for the one device that actually roams. The mesh distributes a nameserver group scoping the fleet's own internal DNS zones (see docs/dns-naming.md) to a dedicated internal resolver, reachable over the mesh itself — but only to the admin group, and marked non-primary so it never competes with a peer's normal DNS for everything else. On the one peer that runs with mesh DNS enabled, those specific internal zones resolve through that mesh path; every other name still goes through whatever resolver the peer would normally use. On-LAN, the directly-connected network resolver still wins for those same zones regardless — this only matters, and only activates, off-LAN, which is exactly the one condition it exists for.

This is the real, live peer roster, not a claimed enrolment count:

Zero-trust Policy: deny-by-default, not just present

The mesh ships with a built-in allow-everything default policy — the first real security decision here was turning it off. With it disabled, no traffic crosses the mesh interface at all unless a specific policy grants it; LAN traffic is completely untouched by this, since the policy only governs the overlay interface, not the physical network.

Groups carry the actual access model:

Group	Role
Admin	Full fleet access — the policy source for everything below
Servers	Every fleet host — coarse policy destination
Endpoints	Personal devices with no fleet access by default — on the mesh for connectivity, not for reach
Functional sub-groups	Hypervisors / infra / Docker hosts / storage — each server is in exactly one, no policy on them yet, there so future rules can be scoped precisely instead of broadly

Policies, all sourced from the admin group and one-directional: SSH and hypervisor-management ports to servers (an explicit port list — the platform's own API rejects ranges, so nothing here is a lazy wildcard), ICMP for diagnostics, and DNS specifically to the infra sub-group to support split-DNS for the roaming device. A posture check gates admin-level access behind a minimum agent version and kernel version — an out-of-date or unpatched peer simply doesn't qualify for the policy regardless of which group it's in. Login expiration applies only to the human-authenticated peer; the fleet's own service-key peers don't expire on a timer, since there's no human session behind them to expire.

This is the real policy list, not a description of the intent — the built-in allow-all sitting disabled at the top, every real rule beneath it scoped to a specific source, direction, and an explicit port list rather than "allow the group and hope":

This is the mesh actually enforcing that model, not a claimed state — a real enrolled peer's own live status:

$ netbird status --detail
Peers detail:
 [peer]:
  NetBird IP: [overlay address]
  Status: Idle
  -- detail --
  Last WireGuard handshake: -
  ...
Events:
  [INFO] SYSTEM
    Message: Network map updated
    Time: 5 minutes, 7 seconds ago

The network map updating on its own, unprompted, on a peer that isn't even the control plane, is the mesh's policy engine actually pushing live state to every connected peer — not a dashboard claiming a policy exists.

11 KiB Raw Blame History