From 2d6be372718c800c0c5b0a3e536849868b6e0f1b Mon Sep 17 00:00:00 2001 From: thesabear Date: Mon, 31 Aug 2026 23:59:36 +0300 Subject: [PATCH] iac-pipeline.md: update for new Terraform LXC support Terraform gained a second provider (bpg/proxmox) alongside the existing telmate/proxmox, via a new containers.tf / proxmox_virtual_environment_container resource - real, live-verified against the actual repo files. Corrects the doc's previous "Terraform only manages virtual machines... cannot create or manage LXC containers" claim, which is no longer true. - Terraform section: VM/LXC comparison table, real containers.tf excerpt, explains why LXCs need an extra pct-exec bootstrap step (stock template has no sshd, since it never went through Packer) - WARNING section: new paragraph noting the LXC-provisioning gap is closed for new hosts going forward (the three existing exceptions stay as preserved history, not retroactively fixed) - forward-links to docs/netbird-zero-trust.md (next commit) - Fixed now-stale section heading ("one resource type" -> "two resource types") - Sanitize-checked: no internal IPs/domains --- docs/iac-pipeline.md | 43 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 40 insertions(+), 3 deletions(-) diff --git a/docs/iac-pipeline.md b/docs/iac-pipeline.md index 56d2779..80d0ae3 100644 --- a/docs/iac-pipeline.md +++ b/docs/iac-pipeline.md @@ -31,12 +31,47 @@ logvol / --fstype=xfs --name=lv_root --vgname=rocky --grow --siz > [!TIP] > Two things happen at image-build time that aren't part of the pipeline's "steady state" and are easy to miss reading only the diagram above. First, the kickstart deliberately leaves SSH in a **permissive** state during the build — `PermitRootLogin yes` and `PasswordAuthentication yes` — because the build environment isn't reachable from anywhere except the hypervisor doing the build, and that window closes for good once the compliance stage's `ssh_hardening` role runs on a provisioned host. Second, Packer's own post-install provisioners create both fleet identities right there in the image — a personal admin account, and the dedicated `ansible` service account used for all automation — and the admin account's setup script ends by writing a sudoers rule that explicitly **denies it permission to run `ansible`/`ansible-playbook` at all**. Automation only ever runs as the service account; that's not a convention someone has to remember, it's baked into every host from before it's even provisioned. -## Terraform: one resource type, tag generation as the real output +## Terraform: two resource types, tag generation as the real output > [!TIP] > Every VM is one entry in a single map — VM ID, target node, template, resources, network, disk, and its own tag list. Two supporting pieces do the real connective work: a wait step that blocks completion until the new VM actually answers SSH as the automation identity (so nothing downstream races a host that isn't up yet), and a template render that writes that VM's tag list directly into Ansible's own host-variable tree. **This tag file is the actual link between provisioning and configuration** — the tags Terraform assigns to a VM are the exact same tags every later Ansible stage (including the security hardening in [`docs/security-monitoring.md`](security-monitoring.md) and the observability rollout in [`docs/observability.md`](observability.md)) matches against. Get this file wrong and every downstream stage silently configures the wrong thing for that host — which is why it's generated, not hand-written. -Terraform only manages virtual machines, deliberately — containers are handled differently (below), a scope boundary that avoids fighting Terraform's own resource model for something it wasn't built to represent well. +Terraform manages both VMs and LXCs now — two separate resource types, one per guest kind, coexisting in one config with no shared state between them: + +| | VMs | LXCs | +|---|---|---| +| Provider | `telmate/proxmox` | `bpg/proxmox` | +| Resource | `proxmox_vm_qemu` | `proxmox_virtual_environment_container` | +| Source image | the `rocky10-cis` Packer template — hardening, `sshd`, both fleet identities pre-baked | stock Rocky 10 template — root-only, no `sshd` | +| Ansible group | `rocky_vm` | `rocky_lxc` | + +The LXC path is genuinely newer and mirrors the VM flow step-for-step, but has to close a gap the VM path never has: the stock container template has no `sshd` and no fleet users at all, since it never went through Packer. A `pct exec` bootstrap step runs first — installing `sshd` and creating both fleet identities with their keys, the same baseline Packer bakes into the VM image at build time instead of at provision time — before the same wait-for-SSH and tag-file-render steps every VM already gets. This is the real resource definition, not a paraphrase: + +``` +$ head -20 containers.tf +resource "proxmox_virtual_environment_container" "ct" { + provider = proxmox-bpg + for_each = var.containers + + node_name = each.value.target_node + vm_id = each.value.vmid + unprivileged = each.value.unprivileged + start_on_boot = each.value.onboot + started = true + + tags = each.value.tags + + cpu { + cores = each.value.cores + } + + memory { + dedicated = each.value.memory + swap = each.value.swap + } +``` + +An empty `var.containers` makes this file completely inert — nothing created, no effect — the same "opt-in, not a parallel system that has to be kept in sync" pattern as every other tag-driven piece of this pipeline. This is the actual template that renders into every host's tag file — the literal source, not a description of its fields: @@ -133,6 +168,8 @@ Compliance hardening runs as a set of independently-tagged sub-roles, selected p > Two hosts don't follow the pipeline above cleanly, and that history is preserved rather than erased. Both were originally provisioned as Terraform-managed virtual machines, then at some point converted directly into LXC containers outside Terraform's management — a resource-type mismatch that Terraform's VM-only tooling genuinely can't represent (LXCs aren't `proxmox_vm_qemu` resources). Rather than deleting their old Terraform entries when this happened, they were commented out, preserving the historical record of how these hosts actually came to exist rather than presenting a cleaner story than what's true. Ansible still configures both today through the same tag-driven mechanism as every VM — only the initial provisioning step is now manual instead of automated. This is the kind of real-world messiness that a from-scratch design document never mentions but that every infrastructure actually accumulates over time — worth showing, not hiding. > > A third host breaks the model in a genuinely different way — not a variation on the same story, a different one entirely. Proxmox Backup Server runs as the vendor's own official Debian-based appliance image, deployed by hand and never a Terraform resource at all — there's no commented-out entry to preserve, because there was never a Terraform-managed version of it to convert away from. It still ended up on the tag-driven Ansible side of the pipeline: wired into its own inventory group and given a hand-written tag list after the fact, since compliance only ever checks whether a tag is present, not how a host arrived in the inventory. It's also the fleet's only non-Rocky host, which surfaced a real gap in one of the 9 hardening sub-roles above: `password_policy` configures PAM lockout behavior, and the ordering it assumes is Rocky's own — safe on every other host, but wrong on this one until it was fixed at the role level and explicitly re-validated. +> +> None of these three exceptions were retroactively fixed once the LXC-provider gap closed — the point of preserving history is that it stays history, not that it gets quietly cleaned up the moment it becomes possible to. But the gap itself is real and worth naming honestly too: for any *new* LXC going forward, this entire honest-exception story no longer applies. The fleet's [zero-trust mesh overlay](netbird-zero-trust.md) is the first host actually built through the LXC-native path above, start to finish, with no manual step anywhere in its provisioning. --- -Related: [`docs/secrets-and-pki.md`](secrets-and-pki.md), [`docs/security-monitoring.md`](security-monitoring.md), [`docs/disaster-recovery.md`](disaster-recovery.md) +Related: [`docs/secrets-and-pki.md`](secrets-and-pki.md), [`docs/security-monitoring.md`](security-monitoring.md), [`docs/disaster-recovery.md`](disaster-recovery.md), [`docs/netbird-zero-trust.md`](netbird-zero-trust.md)