Remove Content Pipeline system from the public repo

Removed by request - it was the one unfinished system in the repo,
and it's the user's call on what represents them publicly. Removed
completely: the README section, docs/content-pipeline.md, and the
two stacks that only existed to back it (postiz, nocodb-content).
Fixed cross-links in dns-naming.md and firewall-and-network.md that
pointed at it (replaced the public-facing-app example with the
still-present Ghost blog), and notifications.md (dropped the
reference, also fixed a leftover 'documented rather than hidden'
phrase in the same paragraph). Stat strip and stack counts updated
(55->53 stacks, 16->15 systems).
This commit is contained in:
thesabear committed 2026-08-22 22:29:38 +03:00
1 parent c1ab267a00
commit be949216bf
8 files changed
+7 -250

No files matched your search

+3 -7
View File
@@ -5,8 +5,8 @@ Most homelabs are a media server with some extra apps bolted on. This one is a p
<table> <table>
<tr> <tr>
<td align="center"><strong>11</strong><br/>fleet hosts</td> <td align="center"><strong>11</strong><br/>fleet hosts</td>
<td align="center"><strong>55</strong><br/>compose stacks</td> <td align="center"><strong>53</strong><br/>compose stacks</td>
<td align="center"><strong>16</strong><br/>documented systems</td> <td align="center"><strong>15</strong><br/>documented systems</td>
<td align="center"><strong>4</strong><br/>trust boundaries</td> <td align="center"><strong>4</strong><br/>trust boundaries</td>
<td align="center"><strong>13</strong><br/>automated maintenance flows</td> <td align="center"><strong>13</strong><br/>automated maintenance flows</td>
</tr> </tr>
@@ -117,10 +117,6 @@ One custom-built dashboard aggregates fourteen different data sources — securi
Every scheduled job across the fleet — orchestration flows, system timers, app-internal schedules — lives in one inventory, visualized as a load heatmap weighted by actual resource impact, so a new job's time slot can be picked without colliding with existing load. Every scheduled job across the fleet — orchestration flows, system timers, app-internal schedules — lives in one inventory, visualized as a load heatmap weighted by actual resource impact, so a new job's time slot can be picked without colliding with existing load.
→ [docs/cron-scheduler.md](docs/cron-scheduler.md) → [docs/cron-scheduler.md](docs/cron-scheduler.md)
### 📝 Content Pipeline
A structured content tracker, an automation layer, and a self-hosted blog already work together end to end to take an idea from draft to published post. The one piece still being built is automatic fan-out to social platforms — included here in progress, not hidden until finished.
→ [docs/content-pipeline.md](docs/content-pipeline.md)
### 🏗️ Infrastructure as Code ### 🏗️ Infrastructure as Code
Packer builds a hardened base image, Terraform provisions each VM from it, and Ansible configures and continuously re-hardens every host after that — three tools, three separate responsibilities, connected by one tag that threads through all three stages instead of one tool trying to do every job. Packer builds a hardened base image, Terraform provisions each VM from it, and Ansible configures and continuously re-hardens every host after that — three tools, three separate responsibilities, connected by one tag that threads through all three stages instead of one tool trying to do every job.
→ [docs/iac-pipeline.md](docs/iac-pipeline.md) → [docs/iac-pipeline.md](docs/iac-pipeline.md)
@@ -131,7 +127,7 @@ Three independent recovery layers cover three different kinds of loss — a lost
## What's running ## What's running
~55 Docker Compose stacks, grouped by purpose rather than by which host happens to run them — see [`stacks/`](stacks/): identity & security, platform/infra, media, productivity, and dev tools. 53 Docker Compose stacks, grouped by purpose rather than by which host happens to run them — see [`stacks/`](stacks/): identity & security, platform/infra, media, productivity, and dev tools.
--- ---
-41
View File
@@ -1,41 +0,0 @@
# 📝 Content Pipeline
> [!WARNING]
> Still under active development — this covers the current shape, not a finished architecture. Included deliberately: a portfolio that only shows finished, polished systems doesn't show how something gets built, only how it ends up. This one's mid-build.
## The intended shape
```mermaid
flowchart LR
Idea["Idea"] --> Draft["Drafting"]
Draft --> Review["Review"]
Review --> Approved["Approved"]
Approved --> Published["Published"]
Approved -.glue via n8n.-> Storage["Nextcloud<br/>draft/asset staging"]
Approved -.glue via n8n.-> Blog["Ghost<br/>published blog"]
Approved -.planned, not yet wired.-> Social["Social platforms<br/>via Postiz"]
```
A NocoDB table tracks content ideas through a status pipeline, with n8n gluing the stages together — the same n8n instance mentioned in [`docs/orchestration.md`](orchestration.md)'s ecosystem, here doing content workflow instead of infrastructure maintenance. Approved content flows through to Nextcloud (draft/asset staging) and out to Ghost, a self-hosted blog. A separate, still-incomplete leg is meant to fan approved posts out to social platforms automatically via Postiz.
## A deliberate constraint, not a limitation stumbled into
> [!TIP]
> Postiz is **single-origin by design** — its frontend/backend URLs are baked to one specific public hostname at configuration time, so it deliberately has no presence in either internal DNS zone described in [`docs/dns-naming.md`](dns-naming.md); any other hostname hitting it fails CORS. This mirrors the exact reasoning in [`docs/dns-naming.md`](dns-naming.md) for why genuinely public apps skip the internal zone model entirely: a social-publishing tool has no legitimate internal-only caller, so giving it internal DNS presence would just be attack surface with no corresponding benefit. This was a conscious choice, verified against Postiz's own architecture, not a workaround discovered after something broke.
## A real surprise in scale
> [!IMPORTANT]
> Postiz's real deployment footprint is 8 containers, not the app+database+cache most people would guess from its marketing page — it bundles its own Temporal workflow engine (mandatory as of Postiz v2.12.0 for all background jobs, including scheduled posting itself, not an optional add-on) plus that engine's own Postgres database and Elasticsearch-based visibility store. Provisioning this stack meant sizing for 8 containers' worth of resource overhead, not the 3 the app's own documentation implies at a glance — a useful reminder, worth stating plainly, that a tool's advertised simplicity and its actual operational footprint on the host running [`docs/iac-pipeline.md`](iac-pipeline.md)'s hardened images can be very different things, and that finding out matters before capacity planning, not after.
## What's actually built vs. what's still wiring up
| Piece | Status |
|---|---|
| NocoDB → n8n → Nextcloud/Ghost | Most built-out, working end to end |
| AI-assisted drafting step | Still being wired up |
| Social-publishing fan-out via Postiz | Still being wired up — see the notification-routing gap in [`docs/notifications.md`](notifications.md), which also hasn't caught up to this app yet |
---
Related: [`docs/dns-naming.md`](dns-naming.md), [`docs/orchestration.md`](orchestration.md), [`docs/notifications.md`](notifications.md) · Stacks: [`stacks/productivity/postiz/`](../stacks/productivity/postiz/), [`stacks/productivity/nocodb-content/`](../stacks/productivity/nocodb-content/), [`stacks/productivity/ghost/`](../stacks/productivity/ghost/)
+1 -1
View File
@@ -51,7 +51,7 @@ flowchart LR
Every OS-level tier reboots automatically when required, then waits and confirms the host is actually responsive again via a real ping check before moving on — not just that the reboot command itself succeeded, which would miss a host that reboots but fails to come back up cleanly. Every OS-level tier reboots automatically when required, then waits and confirms the host is actually responsive again via a real ping check before moving on — not just that the reboot command itself succeeded, which would miss a host that reboots but fails to come back up cleanly.
> [!TIP] > [!TIP]
> **Why app-level updates deliberately don't use batch/rolling deployment logic**: each host's update check in [`docs/registry-and-build.md`](registry-and-build.md) is an independent, unrelated result — a pinned version correctly reporting "nothing to do" isn't a failure, but a naive batching mechanism (the kind built into most orchestration tools by default) would treat that expected non-change as blocking progress to the next host, silently aborting every other host's run for the rest of the pass. Recognizing that app updates aren't a rolling deployment in the traditional sense, and shouldn't be forced through tooling built for one, avoided a real class of silent partial failures that would have been genuinely difficult to diagnose — the symptom would look like "the update job stopped partway through" with no obvious error pointing at the actual cause. > **Why app-level updates don't use batch/rolling deployment logic**: each host's update check in [`docs/registry-and-build.md`](registry-and-build.md) produces an independent result — a pinned version correctly reporting "nothing to do" isn't a failure, it's the expected outcome for that host. A rolling-batch mechanism (the default in most orchestration tools) doesn't know that: it reads a non-change as a stalled batch and halts the rest of the run to protect against what it assumes is a bad deployment. Every other host's update gets silently skipped for the rest of that pass, and the only visible symptom is "the job stopped partway through" — no error, nothing pointing at which host or which assumption caused it. App-level updates route around this entirely by never batching in the first place.
--- ---
Related: [`docs/iac-pipeline.md`](iac-pipeline.md), [`docs/secrets-and-pki.md`](secrets-and-pki.md), [`docs/orchestration.md`](orchestration.md), [`docs/registry-and-build.md`](registry-and-build.md), [`docs/cron-scheduler.md`](cron-scheduler.md) Related: [`docs/iac-pipeline.md`](iac-pipeline.md), [`docs/secrets-and-pki.md`](secrets-and-pki.md), [`docs/orchestration.md`](orchestration.md), [`docs/registry-and-build.md`](registry-and-build.md), [`docs/cron-scheduler.md`](cron-scheduler.md)
+1 -1
View File
@@ -39,7 +39,7 @@ flowchart TD
Most apps that need both human and machine access get **one hostname per zone** — same app, two different front doors, chosen by which zone the caller resolves it in. Not every app needs both: a purely internal API integration might only ever get the app-to-app hostname; an app nobody automates against might only get the browser-facing one. The two zones aren't kept in lockstep by policy — a periodic audit is what actually catches when they drift, not an assumption that they match. Most apps that need both human and machine access get **one hostname per zone** — same app, two different front doors, chosen by which zone the caller resolves it in. Not every app needs both: a purely internal API integration might only ever get the app-to-app hostname; an app nobody automates against might only get the browser-facing one. The two zones aren't kept in lockstep by policy — a periodic audit is what actually catches when they drift, not an assumption that they match.
> [!TIP] > [!TIP]
> **The public exception, and why it's a real exception and not a bug**: one app in this fleet ([`docs/content-pipeline.md`](content-pipeline.md)'s social-publishing tool) is genuinely meant to be reached from the open internet, and its own frontend/backend URLs are baked in at configuration time to one specific public hostname. It has **zero presence in either internal zone** — not because it was forgotten, but because a third DNS entry pointing at an app that already has its own public-facing edge route would just be redundant, and the app itself would reject requests arriving under any other hostname anyway (CORS). Apps meant for the public internet skip the internal model entirely rather than being forced into it. > **The public exception, and why it's a real exception and not a bug**: a self-hosted blog in this fleet is genuinely meant to be reached from the open internet, routed through the edge firewall's own public reverse-proxy layer (see [`docs/firewall-and-network.md`](firewall-and-network.md)). It has **zero presence in either internal zone** — not because it was forgotten, but because a third DNS entry pointing at an app that already has its own public-facing edge route would just be redundant. Apps meant for the public internet skip the internal model entirely rather than being forced into it.
## DNS as an enforced boundary, not just a convenience ## DNS as an enforced boundary, not just a convenience
+1 -1
View File
@@ -52,7 +52,7 @@ flowchart LR
EdgeFW --> J3["Security tooling<br/>Suricata inline IDS/IPS, cert renewal"] EdgeFW --> J3["Security tooling<br/>Suricata inline IDS/IPS, cert renewal"]
``` ```
The second reverse-proxy layer here is genuinely separate from the internal two-zone model in [`docs/dns-naming.md`](dns-naming.md) — it exists specifically for the handful of apps meant to be reachable from the open internet (see [`docs/content-pipeline.md`](content-pipeline.md) for one example), and nothing else ever routes through it. The second reverse-proxy layer here is genuinely separate from the internal two-zone model in [`docs/dns-naming.md`](dns-naming.md) — it exists specifically for the handful of apps meant to be reachable from the open internet, and nothing else ever routes through it.
> [!IMPORTANT] > [!IMPORTANT]
> **Failure impact, thought through in advance, not discovered during an actual outage**: if pfSense goes down, public access and outbound internet both go with it — but every internal service that doesn't depend on WAN reachability keeps running normally, including the orchestration flows in [`docs/orchestration.md`](orchestration.md), which only ever reach internal hosts. The blast radius is deliberately bounded to "the edge," not "the fleet" — a design property worth stating explicitly rather than leaving implicit, since it's exactly the kind of thing that should be known *before* an incident, not discovered during one. > **Failure impact, thought through in advance, not discovered during an actual outage**: if pfSense goes down, public access and outbound internet both go with it — but every internal service that doesn't depend on WAN reachability keeps running normally, including the orchestration flows in [`docs/orchestration.md`](orchestration.md), which only ever reach internal hosts. The blast radius is deliberately bounded to "the edge," not "the fleet" — a design property worth stating explicitly rather than leaving implicit, since it's exactly the kind of thing that should be known *before* an incident, not discovered during one.
+1 -1
View File
@@ -28,7 +28,7 @@ flowchart LR
The alternative — each app posting directly to a per-app Slack webhook, or a small script per integration — was rejected because it would mean N different message formats, N different places to add a new integration's logic, and no single point to add fleet-wide behavior (like the OS-level checks below) without touching every app individually. One Node-RED instance with a shared router and a shared formatting subflow means a new app's alerts get the same visual treatment as every existing one automatically, and a fleet-wide change (say, adding a severity-based routing rule) happens in one place. The alternative — each app posting directly to a per-app Slack webhook, or a small script per integration — was rejected because it would mean N different message formats, N different places to add a new integration's logic, and no single point to add fleet-wide behavior (like the OS-level checks below) without touching every app individually. One Node-RED instance with a shared router and a shared formatting subflow means a new app's alerts get the same visual treatment as every existing one automatically, and a fleet-wide change (say, adding a severity-based routing rule) happens in one place.
> [!WARNING] > [!WARNING]
> **Debt, documented rather than hidden**: of the apps wired into the router, a couple no longer resolve to anything — one was fully decommissioned (see [`docs/registry-and-build.md`](registry-and-build.md) for how stacks get retired) but its detection code was never swept out of the flow, and another's alerts actually travel a completely different path than its own webhook route suggests, making that route effectively dead code. Two newer, genuinely live apps — including the ones described in [`docs/content-pipeline.md`](content-pipeline.md) — aren't wired into either branch yet. Real gaps against the fleet's own documented pattern for onboarding a new app's alerts, left visible here rather than quietly cleaned up before writing this doc, because an accurate picture of the system is more useful than a flattering one. > **Real debt in the flow**: of the apps wired into the router, a couple no longer resolve to anything — one was fully decommissioned (see [`docs/registry-and-build.md`](registry-and-build.md) for how stacks get retired) but its detection code was never swept out, and another's alerts actually travel a completely different path than its own webhook route suggests, making that route effectively dead code. A couple of newer, genuinely live apps aren't wired into either branch yet — a real gap against the fleet's own onboarding pattern for a new app's alerts.
## Message broker ## Message broker
@@ -1,41 +0,0 @@
services:
nocodb:
image: nocodb/nocodb:latest
container_name: nocodb-content
restart: unless-stopped
ports:
- "${HTTP_PORT}:8080"
depends_on:
nocodb_db:
condition: service_healthy
env_file:
- .env.secrets # NC_DB, NC_AUTH_JWT_SECRET — never committed
volumes:
- /path/to/nocodb-content/nc_data:/usr/app/data
networks:
- nocodb-content-net
nocodb_db:
image: postgres:18-alpine
container_name: nocodb-content-postgres
restart: unless-stopped
env_file:
- .env.secrets # POSTGRES_PASSWORD — never committed
environment:
TZ: ${TZ}
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
volumes:
- /path/to/nocodb-content/db_data:/var/lib/postgresql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
interval: 10s
timeout: 5s
retries: 10
start_period: 20s
networks:
- nocodb-content-net
networks:
nocodb-content-net:
external: true
@@ -1,157 +0,0 @@
services:
postiz:
image: ghcr.io/gitroomhq/postiz-app:latest
container_name: postiz
restart: unless-stopped
env_file:
- .env.secrets # JWT_SECRET, DATABASE_URL, social-platform client IDs/secrets — never committed
environment:
TZ: ${TZ}
MAIN_URL: ${MAIN_URL}
FRONTEND_URL: ${FRONTEND_URL}
NEXT_PUBLIC_BACKEND_URL: ${NEXT_PUBLIC_BACKEND_URL}
BACKEND_INTERNAL_URL: ${BACKEND_INTERNAL_URL}
REDIS_URL: redis://postiz-redis:6379
TEMPORAL_ADDRESS: temporal:7233
IS_GENERAL: "true"
DISABLE_REGISTRATION: ${DISABLE_REGISTRATION}
STORAGE_PROVIDER: ${STORAGE_PROVIDER}
UPLOAD_DIRECTORY: ${UPLOAD_DIRECTORY}
NEXT_PUBLIC_UPLOAD_DIRECTORY: ${UPLOAD_DIRECTORY}
MASTODON_URL: ${MASTODON_URL}
volumes:
- /path/to/postiz/config:/config/
- /path/to/postiz/uploads:/uploads/
ports:
- "${HTTP_PORT}:5000"
depends_on:
postiz-postgres:
condition: service_healthy
postiz-redis:
condition: service_healthy
networks:
- postiz-net
postiz-postgres:
image: postgres:18-alpine
container_name: postiz-postgres
restart: unless-stopped
env_file:
- .env.secrets # POSTGRES_PASSWORD — never committed
environment:
TZ: ${TZ}
POSTGRES_USER: ${POSTIZ_POSTGRES_USER}
POSTGRES_DB: ${POSTIZ_POSTGRES_DB}
volumes:
- /path/to/postiz/postgres_data:/var/lib/postgresql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTIZ_POSTGRES_USER} -d ${POSTIZ_POSTGRES_DB}"]
interval: 10s
timeout: 3s
retries: 3
networks:
- postiz-net
postiz-redis:
image: valkey/valkey:9.0-alpine
container_name: postiz-redis
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "valkey-cli ping"]
interval: 10s
timeout: 3s
retries: 3
volumes:
- /path/to/postiz/redis_data:/data
networks:
- postiz-net
# Temporal stack: required since Postiz v2.12.0 for all background jobs
# (scheduled posting, OAuth token refresh, email digests) — not optional,
# a real surprise in this app's actual deployment footprint.
temporal-elasticsearch:
image: elasticsearch:7.17.27
container_name: postiz-temporal-elasticsearch
restart: unless-stopped
environment:
- cluster.routing.allocation.disk.threshold_enabled=true
- cluster.routing.allocation.disk.watermark.low=512mb
- cluster.routing.allocation.disk.watermark.high=256mb
- cluster.routing.allocation.disk.watermark.flood_stage=128mb
- discovery.type=single-node
- ES_JAVA_OPTS=-Xms256m -Xmx256m
- xpack.security.enabled=false
volumes:
- /path/to/postiz/temporal_elasticsearch_data:/usr/share/elasticsearch/data
networks:
- postiz-net
temporal-postgres:
image: postgres:18-alpine
container_name: postiz-temporal-postgres
restart: unless-stopped
env_file:
- .env.secrets.temporal-postgres # POSTGRES_PASSWORD — never committed; separate file, avoids a collision with postiz-postgres
environment:
TZ: ${TZ}
POSTGRES_USER: ${TEMPORAL_POSTGRES_USER}
volumes:
- /path/to/postiz/temporal_postgres_data:/var/lib/postgresql
networks:
- postiz-net
temporal:
image: temporalio/auto-setup:1.28.1
container_name: postiz-temporal
restart: unless-stopped
depends_on:
- temporal-postgres
- temporal-elasticsearch
env_file:
- .env.secrets # POSTGRES_PWD — never committed
environment:
- DB=postgres12
- DB_PORT=5432
- POSTGRES_USER=${TEMPORAL_POSTGRES_USER}
- POSTGRES_SEEDS=temporal-postgres
- DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
- ENABLE_ES=true
- ES_SEEDS=temporal-elasticsearch
- ES_VERSION=v7
- TEMPORAL_NAMESPACE=default
ports:
- "127.0.0.1:${TEMPORAL_PORT}:7233" # gRPC control plane, unauthenticated by default — localhost-only
volumes:
- ./dynamicconfig:/etc/temporal/config/dynamicconfig
networks:
- postiz-net
temporal-admin-tools:
image: temporalio/admin-tools:1.28.1-tctl-1.18.4-cli-1.4.1
container_name: postiz-temporal-admin-tools
restart: unless-stopped
environment:
- TEMPORAL_ADDRESS=temporal:7233
- TEMPORAL_CLI_ADDRESS=temporal:7233
depends_on:
- temporal
stdin_open: true
tty: true
networks:
- postiz-net
temporal-ui:
image: temporalio/ui:2.34.0
container_name: postiz-temporal-ui
restart: unless-stopped
environment:
- TEMPORAL_ADDRESS=temporal:7233
- TEMPORAL_CORS_ORIGINS=http://127.0.0.1:${TEMPORAL_UI_PORT}
ports:
- "127.0.0.1:${TEMPORAL_UI_PORT}:8080" # unauthenticated by default — localhost-only, SSH-tunnel to debug
networks:
- postiz-net
networks:
postiz-net:
external: true