capitalize headings

This commit is contained in:
thesabear committed 2026-09-01 17:59:02 +03:00
1 parent 4c1b23a69c
commit f20ed10632
16 files changed
+65 -65

No files matched your search

+6 -6
View File
@@ -3,11 +3,11 @@
> [!NOTE]
> Mosquitto (MQTT) and a single Node-RED instance are the backbone here — two independent fan-in branches, app webhooks and OS-level host alerts, converging on one flow and one shared formatter before reaching Slack. The point isn't the individual pieces; it's that every alert reads the same regardless of where it came from.
## Fan-in, fan-out
## Fan-in, Fan-out
![Three independent fan-in sources converging on one shared formatting subflow: 15 apps over HTTP webhook go through the App Router and Switch to a per-app mapper, Kopia/Bazarr post to their own dedicated endpoints and bypass the router entirely with their own mapper, and every host publishes over MQTT into a topic-agnostic Inventory Parse function — all three land on the same subflow before reaching Slack](../assets/notifications-fan-in-fan-out.svg)
## Why one Node-RED instance instead of per-app scripts
## Why One Node-RED Instance Instead Of Per-App Scripts
The alternative — each app posting directly to a per-app Slack webhook, or a small script per integration — was rejected because it would mean N different message formats, N different places to add a new integration's logic, and no single point to add fleet-wide behavior (like the OS-level checks below) without touching every app individually. One Node-RED instance with a shared router and a shared formatting subflow means a new app's alerts get the same visual treatment as every existing one automatically, and a fleet-wide change (say, adding a severity-based routing rule) happens in one place.
@@ -46,7 +46,7 @@ This is the real flow, not a redrawn version of it — the App Router feeding a
![Real Node-RED canvas: HTTP-in webhook through App Router into a 16-rule Switch function feeding per-app mapper functions (including netbox mapper and kestra mapper, both still wired but dead), MQTT-in through the Inventory Parse Function, and Kopia/Bazarr's own dedicated HTTP-in nodes bypassing the router entirely — all converging on the shared MQTT → Slack subflow](../assets/notifications-node-red-flow.png)
## Message broker
## Message Broker
| | |
|---|---|
@@ -85,11 +85,11 @@ And this is the broker actually being used, not just configured — real connect
> [!IMPORTANT]
> **A real, tracked fragility, worth naming precisely**: the publish identity's password has to stay in sync across three separate places — the Vault-rendered broker credential file, the source value used to render every host's own publish script (via the same tag-driven Ansible model in [`docs/iac-pipeline.md`](iac-pipeline.md)), and a hardcoded copy inside the orchestration engine's own alert-publishing flow definition (not sourced from Vault at all, unlike everything else in [`docs/secret-distribution.md`](secret-distribution.md)). Rotating it means updating all three by hand — a documented, real exception to the "one source of truth" rule that governs every other secret in this fleet, called out explicitly rather than left for someone to discover mid-rotation.
## Detection logic
## Detection Logic
The Node-RED App Router figures out which app sent a webhook by inspecting the payload's shape — an explicit field for some apps, structural pattern-matching for others (Emby, for instance, is identified purely by a nested `data.Server` field only that vendor's payload happens to include, since it never sends an explicit app identifier). Kopia and Bazarr skip the shared router entirely with their own dedicated endpoints, since they post plain text rather than JSON and the shared router's shape-detection logic simply doesn't apply to them.
## Every host, not just apps
## Every Host, Not Just Apps
> [!TIP]
> A dedicated Ansible role puts every fleet host onto this same pipeline for OS-level health, independent of any application-level alert — the same fleet-wide, tag-driven rollout pattern used for the observability agent in [`docs/observability.md`](observability.md) and the compliance hardening in [`docs/security-monitoring.md`](security-monitoring.md). It runs over 20 distinct checks per host — disk thresholds, password/certificate expiry, failed logins (both local and from outside the trusted network), cron/systemd failures, container health, pending reboots, time-sync drift, OOM events, read-only filesystem remounts, and several security-platform-specific checks (Vault audit log anomalies, Vault Agent authentication failures) — deliberately complementary to Wazuh in [`docs/security-monitoring.md`](security-monitoring.md), not duplicating its file-integrity or account monitoring. Splitting OS-health checks from the SOC's own detection engine keeps each system's role narrow enough to actually reason about, instead of one monolithic monitoring tool trying to do both jobs at once.
@@ -103,7 +103,7 @@ The Node-RED App Router figures out which app sent a webhook by inspecting the p
>
> 22 from that count, plus disk-usage thresholds — which don't show up in that grep because its state key is built dynamically per mountpoint, not a fixed name — makes 23. One of those 22 was a genuine KB gap this doc caught mid-audit: crash-looping systemd units (`activating`/`auto-restart` stuck in a loop) never trips the plain failed-units check above it, because a unit that keeps getting restarted never actually reaches systemd's "failed" state. That distinction wasn't theoretical — it's in the script because of a real incident where exactly that happened and produced zero alerts.
## One formatter, every source
## One Formatter, Every Source
Every branch — app webhooks, host-level alerts, and Kopia/Bazarr's own dedicated non-JSON endpoints — converges on one shared subflow before anything reaches Slack. This is the single point where message formatting is standardized, regardless of which of the dozen-plus sources triggered it — a reader in Slack never has to learn a different alert format depending on which system fired it.