diff --git a/assets/secret-distribution-flows.svg b/assets/secret-distribution-flows.svg new file mode 100644 index 0000000..d2eca35 --- /dev/null +++ b/assets/secret-distribution-flows.svg @@ -0,0 +1,4 @@ + + + +
Provisioning Repo
template inventory
Vault Agent Config
on the target host
DATA PLANE — what Vault Agent actually renders
Vault Agent
Certificate Flow
PKI engine, short-lived
App Secret Flow
KV, rendered 0600
Container
reads local disk only
A SEPARATE SYSTEM — no Vault involved
Administrative Secret Flow
ansible-vault encrypted,
committed to git
push, target host pulls
restarts, picks up new template list
decrypted on demand
Orange = control plane (Configuration Distribution Flow) — gates what Vault Agent renders, doesn't touch Vault itself. Purple = data plane (Certificate + App Secret flows) — both run through the same Vault Agent, both consumed locally. Gray = genuinely separate system (Administrative Secret Flow) — no Vault involvement at all, only reachable by whoever holds the ansible-vault passphrase.
 \ No newline at end of file diff --git a/assets/vault-agent-reload.png b/assets/vault-agent-reload.png new file mode 100644 index 0000000..2668e01 Binary files /dev/null and b/assets/vault-agent-reload.png differ diff --git a/docs/secret-distribution.md b/docs/secret-distribution.md index f455889..b5f9e05 100644 --- a/docs/secret-distribution.md +++ b/docs/secret-distribution.md @@ -3,17 +3,7 @@ > [!NOTE] > [`docs/secrets-and-pki.md`](secrets-and-pki.md) covers *what* Vault is and *why* it exists. This doc covers something different and, arguably, harder to get right: the actual contract for how a secret value moves from being written into Vault to being consumed by a running container — and why that contract is deliberately split into four separate flows instead of one universal one. -```mermaid -flowchart TD - Op["Operator"] -->|1: writes cert/secret request| Vault["Vault"] - Vault -->|2: Certificate Flow| Cert["Short-lived cert
rendered to disk"] - Vault -->|3: App Secret Flow| Sec["Rendered secret file
0600, gitignored"] - Repo["Provisioning repo"] -->|4: Config Distribution Flow| Vault - Op2["Operator"] -.5: Administrative Secret Flow
separate system entirely.-> EncFile["ansible-vault encrypted file
in the repo itself"] - - Cert --> Container["Container reads
local disk only"] - Sec --> Container -``` +![Secret distribution flows: Configuration Distribution Flow (orange) is the control plane that gates what Vault Agent renders; Certificate Flow and App Secret Flow (purple) are the data plane running through that same agent; Administrative Secret Flow (gray) is a genuinely separate system with no Vault involvement](../assets/secret-distribution-flows.svg) ## Why four flows instead of one @@ -51,6 +41,10 @@ The obvious simpler design is one universal "secret goes in, secret comes out" p > [!IMPORTANT] > This is the flow with the sharpest failure mode in the whole model: a missing or malformed template referenced by a host's own config crashes that host's **entire** Vault Agent process — every other secret/cert that host was already rendering stops updating too, not just the one broken template. A single YAML typo in one app's template inventory can silently freeze every other app's secret rotation on that same host. This is why a config redeploy always requires confirming the underlying git change actually landed first, not just that the deploy command returned success. +That redeploy-and-restart sequence isn't just a claim — here's a real one, start to finish: Vault Agent restarts, authenticates, renders a fresh `.env.secrets`, and its `reload_command` fires `docker compose up -d`, which recreates and restarts the actual container: + +![A real Vault Agent restart: authenticates, renders triliumnext-mcp's .env.secrets, then its reload_command runs docker compose up -d, recreating and starting the container](../assets/vault-agent-reload.png) + **Shared-secret variant:** not every template is host-scoped — one credential needed by every docker host for its own registry pushes (see [`docs/registry-and-build.md`](registry-and-build.md)) is templated from a single shared path rather than a per-host one. Same mechanics, just a shared source instead of a private one — and it relies on every host's policy already granting read access to that shared path, the same "shared + private" pattern described in [`docs/secrets-and-pki.md`](secrets-and-pki.md)'s access model. ## 4️⃣ Administrative Secret Flow — a deliberately separate, simpler system @@ -64,5 +58,8 @@ The obvious simpler design is one universal "secret goes in, secret comes out" p > [!TIP] > **Why two systems, not one — the actual reasoning, not just the rule**: Vault KV is built for runtime/service/pipeline secrets a running container needs continuously and automatically. `ansible-vault` is built for secrets used by tooling that *operates on* the fleet rather than secrets *consumed by* the fleet's own running services — a Gitea admin token used only during a playbook run, for instance, doesn't need automatic rotation, per-host scoping, or a rendered file sitting on disk permanently. Building that into Vault's model would have meant either weakening Vault's per-host isolation for a use case that doesn't need it, or adding a whole parallel "admin secrets" concept inside Vault itself. Using the tool already built for exactly this case (`ansible-vault`, already part of the Ansible toolchain from [`docs/iac-pipeline.md`](iac-pipeline.md)) was less work and a better fit than extending Vault to do a second, different job. +> [!NOTE] +> **One exception to this split, flagged rather than smoothed over**: Technitium's API token moved out of `ansible-vault` into live Vault KV (fetched via a `community.hashi_vault.vault_kv2_get` lookup at playbook-run time) even though it's consumed by an ansible-playbook, not a running service — exactly the case the law above says belongs in `ansible-vault` instead. The admin/runtime split is the general rule here, not an absolute one, and this is the one place it doesn't cleanly apply. + --- Related: [`docs/secrets-and-pki.md`](secrets-and-pki.md), [`docs/iac-pipeline.md`](iac-pipeline.md), [`docs/registry-and-build.md`](registry-and-build.md)