docs: rewrite registry-and-build.md with real evidence and diagram
Live-verified the container_update role's strategy_dockerfile.yml task file exists at the exact path the KB names, stamped the source note. Replaced the mermaid cheap-check flowchart with a real branch-shaped Nextcloud diagram. Added: the real Vault-backed push auth mechanism (Gitea PAT -> Vault Agent -> docker login --password-stdin, same pattern as everywhere else in the fleet), the real docker manifest inspect command instead of a vague "digest check", real app names in the holdouts table instead of generic descriptions (matching how the rest of this repo already names real apps), the anonymous-volume-pruning detail, and a real (redacted) screenshot of the actual Gitea package registry showing genuine self-built container images.
This commit is contained in:
3 files changed
+18
-18
No files matched your search
@@ -0,0 +1,4 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!-- Do not edit this file with editors other than draw.io -->
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" style="background: transparent; background-color: transparent; color-scheme: light dark;" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="912px" height="429px" viewBox="0 0 912 429" content="<mxfile host="embed.diagrams.net" scale="1" border="0"> <diagram name="Cheap Check First Build" id="cheap-check-1"> <mxGraphModel dx="1414" dy="791" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="460" math="0" shadow="0"> <root> <mxCell id="0" /> <mxCell id="1" parent="0" /> <mxCell id="trigger" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#F5F5F5;strokeColor=#666666;fontColor=#333333;fontSize=13;fontStyle=1;shadow=1;" value="Weekly Kestra trigger" vertex="1"> <mxGeometry height="60" width="170" x="30" y="180" as="geometry" /> </mxCell> <mxCell id="check" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontColor=#333333;fontSize=13;fontStyle=1;shadow=1;" value="docker manifest inspect&lt;br&gt;&lt;font style=&quot;font-size: 11px;&quot;&gt;metadata only, no pull&lt;/font&gt;" vertex="1"> <mxGeometry height="70" width="200" x="240" y="175" as="geometry" /> </mxCell> <mxCell id="gate" parent="1" style="rhombus;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;fontColor=#333333;fontSize=12;fontStyle=1;shadow=1;" value="Upstream digest ==&lt;br&gt;fleet registry digest?" vertex="1"> <mxGeometry height="120" width="200" x="480" y="150" as="geometry" /> </mxCell> <mxCell id="nothing" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontColor=#333333;fontSize=13;fontStyle=1;shadow=1;" value="Nothing runs&lt;br&gt;&lt;font style=&quot;font-size: 11px;&quot;&gt;no pull, no build, no push&lt;/font&gt;" vertex="1"> <mxGeometry height="70" width="200" x="480" y="20" as="geometry" /> </mxCell> <mxCell id="pull" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=light-dark(#FFFFFF,#392F1E);strokeColor=#d79b00;fontColor=#333333;fontSize=12;fontStyle=1;shadow=1;" value="Pull base images" vertex="1"> <mxGeometry height="55" width="160" x="770" y="60" as="geometry" /> </mxCell> <mxCell id="build" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=light-dark(#FFFFFF,#392F1E);strokeColor=#d79b00;fontColor=#333333;fontSize=12;fontStyle=1;shadow=1;" value="docker compose build&lt;br&gt;--no-cache" vertex="1"> <mxGeometry height="55" width="160" x="770" y="145" as="geometry" /> </mxCell> <mxCell id="compare" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=light-dark(#FFFFFF,#392F1E);strokeColor=#d79b00;fontColor=#333333;fontSize=12;fontStyle=1;shadow=1;" value="Compare pre/post&lt;br&gt;image IDs" vertex="1"> <mxGeometry height="55" width="160" x="770" y="230" as="geometry" /> </mxCell> <mxCell id="push" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=light-dark(#FFFFFF,#392F1E);strokeColor=#d79b00;fontColor=#333333;fontSize=12;fontStyle=1;shadow=1;" value="Push what&lt;br&gt;actually changed" vertex="1"> <mxGeometry height="55" width="160" x="770" y="315" as="geometry" /> </mxCell> <mxCell id="redeploy" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79Line truncated
|
||||
|
After Width: | Height: | Size: 311 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 247 KiB |
+14
-18
@@ -13,33 +13,29 @@
|
||||
|
||||
Pulling directly from Docker Hub/ghcr at runtime was the status quo before this existed, and it was replaced for a concrete reason: every container pull became a runtime dependency on an external service staying up and staying reachable through the firewall model in [`docs/firewall-and-network.md`](firewall-and-network.md). Building images in-fleet and pushing to a registry that's already inside the trust boundary means a redeploy never depends on an external registry's availability, and the exact image contents are fully within this fleet's own control rather than whatever an upstream maintainer pushed as `:latest` today.
|
||||
|
||||
Pushing to the registry uses the same Vault-backed secrets pattern as everything else in [`docs/secrets-and-pki.md`](secrets-and-pki.md), not a token pasted into a CI config somewhere: a Gitea Personal Access Token scoped to `write:package` only, rendered by Vault Agent to a `0600` file on each building host and piped straight into `docker login --password-stdin` — never typed, never sitting in a config file with broader scope than it needs.
|
||||
|
||||
This is the real registry, not a claim about it — every one of these is a genuine self-built container, not a Docker Hub pull:
|
||||
|
||||
|
||||
|
||||
## The cheap-check-first build decision
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start["Weekly Kestra trigger<br/>see docs/orchestration.md"] --> Check["Cheap digest check<br/>metadata only, no pull"]
|
||||
Check --> Match{"Upstream digest ==<br/>fleet registry digest?"}
|
||||
Match -->|yes| Nothing["Nothing runs<br/>no pull, no build, no push"]
|
||||
Match -->|no| Pull["Pull base images"]
|
||||
Pull --> Build["docker compose build --no-cache"]
|
||||
Build --> Compare["Compare pre/post image IDs"]
|
||||
Compare --> Push["Push what actually changed"]
|
||||
Push --> Redeploy["Redeploy via shared<br/>health-check/rollback path"]
|
||||
```
|
||||
|
||||
|
||||
> [!TIP]
|
||||
> Most stacks use a minimal wrapper Dockerfile around the upstream image rather than a full clone-and-build-from-source setup — simpler, builds near-instantly, can't break from upstream build tooling changing underneath it, while still achieving the real goal: an in-fleet registry with no direct Docker Hub/ghcr dependency at runtime. A handful of multi-build-image stacks (the ones with more than one real upstream component, like the identity provider in [`docs/sso-and-identity.md`](sso-and-identity.md)) skip the cheap check entirely, since a single "upstream digest" comparison doesn't make sense when there's more than one upstream involved.
|
||||
> Most stacks use a minimal `FROM <upstream-image>:${VERSION}` wrapper Dockerfile rather than a full clone-and-build-from-source setup — simpler, builds near-instantly, can't break from upstream build tooling changing underneath it, while still achieving the real goal: an in-fleet registry with no direct Docker Hub/ghcr dependency at runtime. A handful of multi-build-image stacks (the ones with more than one real upstream component, like Paperless-ngx, Ghost, Ollama, and the Grafana stack) skip the cheap check entirely, since a single "upstream digest" comparison doesn't make sense when there's more than one upstream involved.
|
||||
|
||||
## Deliberate holdouts
|
||||
|
||||
Not every stack is on this pattern, and that's a decision, not an oversight — knowing when *not* to apply a pattern uniformly is as much a design skill as building the pattern in the first place:
|
||||
|
||||
| Stack type | Why it's held out |
|
||||
| Stack | Why it's held out |
|
||||
|---|---|
|
||||
| Complex multi-service upstream builds (e.g. the identity provider, a photo-management app, a media transcoding tool) | Not worth replicating a build process someone else already maintains well — the maintenance cost of tracking upstream's own build changes would exceed the benefit |
|
||||
| A docker-socket-proxy style stack | Simple enough that a custom build adds no real value over the upstream image directly |
|
||||
| One database service still on `latest` | Deliberately not pinned to this pattern — low enough stakes that the extra build step isn't worth it |
|
||||
| A productivity suite with pre-existing custom Dockerfiles | Left as-is beyond adding image references — its own DB/cache still float on Docker Hub tags directly, a conscious "don't touch what already works" call |
|
||||
| Authentik, Immich, tdarr | Complex multi-service upstream builds — not worth replicating a build process the upstream project already maintains well; the maintenance cost of tracking their own build changes would exceed the benefit |
|
||||
| docker-socket-proxy | Simple enough that a custom build adds no real value over the upstream image directly |
|
||||
| Kodi's mariadb service | Deliberately still on `latest`, unpinned — low enough stakes that the extra build step isn't worth it |
|
||||
| Nextcloud's own `nextcloud`/`euro-office` services | Pre-existing custom Dockerfiles, left as-is beyond adding `image:` references — its own db/redis still float on Docker Hub tags directly, a conscious "don't touch what already works" call |
|
||||
|
||||
## Trigger discipline
|
||||
|
||||
@@ -48,7 +44,7 @@ Not every stack is on this pattern, and that's a decision, not an oversight —
|
||||
|
||||
## Housekeeping
|
||||
|
||||
Unused local images get pruned automatically at the end of every build pass — for build-and-push-only stacks, the registry is the source of truth, not whatever happens to be sitting on local disk, so nothing is lost by aggressively reclaiming local space immediately.
|
||||
Unused local images get pruned automatically at the end of every build pass — for build-and-push-only stacks, the registry is the source of truth, not whatever happens to be sitting on local disk, so nothing is lost by aggressively reclaiming local space immediately. A volume prune runs right after it, for a less obvious reason: build-and-push-only database stacks declare no `volumes:` block of their own, but the base image's own `VOLUME` directive still leaves an anonymous volume behind on every single build regardless — always safe to reclaim immediately, since the prune only ever removes volumes with zero remaining container references.
|
||||
|
||||
---
|
||||
Related: [`docs/iac-pipeline.md`](iac-pipeline.md), [`docs/orchestration.md`](orchestration.md), [`docs/sso-and-identity.md`](sso-and-identity.md) · Stacks: [`stacks/platform/gitea/`](../stacks/platform/gitea/), [`stacks/platform/kestra/`](../stacks/platform/kestra/)
|
||||
Reference in new issue
Block a user