docs: fix notifications.md drift (netbox cleanup, switch rule count)

Live-checked the flow export directly: NetBox's dead switch rule and
mapper function have actually been removed since this doc was written
(16->15 switch rules, 15->14 routed apps) - the doc's own 'debt' claim
was half-stale, since only Kestra's dead webhook route remains.
Corrected all four places this count appeared (routing-table example,
two screenshot captions, one diagram's alt-text). Also fixed the
mqtt-monitor.sh check-count example: raw grep now returns 23 due to a
companion state file (failed_units_current) for an existing check, not
a genuinely new 23rd check - reworded so the total (23) stays accurate
without implying a new check was silently added. Mosquitto broker
config/passwords/ACL and the Inventory Parse Function code both
confirmed unchanged.
This commit is contained in:
automation-bot committed 2026-09-19 09:18:35 +03:00
1 parent 4d7ab28876
commit 8258339daf
1 file changed
+10 -10
+10 -10
View File
@@ -1,6 +1,6 @@
<!-- verified: 2026-09-06 <!-- verified: 2026-09-19
automation-vm@7c057de : mosquitto automation-vm@7c057de : mosquitto
iac@49ff427 : ansible/roles/monitoring/mqtt_client iac@f770e1c : ansible/roles/monitoring/mqtt_client
trilium : tt4idCVJPpZM, EAU5LUwFFQ0E, rGfAb4CddKMi, otSPaU6IfjlP, klrmnlh3Wrnd trilium : tt4idCVJPpZM, EAU5LUwFFQ0E, rGfAb4CddKMi, otSPaU6IfjlP, klrmnlh3Wrnd
--> -->
@@ -11,14 +11,14 @@
## Fan-in, Fan-out ## Fan-in, Fan-out
![Three independent fan-in sources converging on one shared formatting subflow: 15 apps over HTTP webhook go through the App Router and Switch to a per-app mapper, Kopia/Bazarr post to their own dedicated endpoints and bypass the router entirely with their own mapper, and every host publishes over MQTT into a topic-agnostic Inventory Parse function — all three land on the same subflow before reaching Slack](../assets/notifications-fan-in-fan-out.svg) ![Three independent fan-in sources converging on one shared formatting subflow: 14 apps over HTTP webhook go through the App Router and Switch to a per-app mapper, Kopia/Bazarr post to their own dedicated endpoints and bypass the router entirely with their own mapper, and every host publishes over MQTT into a topic-agnostic Inventory Parse function — all three land on the same subflow before reaching Slack](../assets/notifications-fan-in-fan-out.svg)
## Why One Node-RED Instance Instead Of Per-App Scripts ## Why One Node-RED Instance Instead Of Per-App Scripts
The alternative — each app posting directly to a per-app Slack webhook, or a small script per integration — was rejected because it would mean N different message formats, N different places to add a new integration's logic, and no single point to add fleet-wide behavior (like the OS-level checks below) without touching every app individually. One Node-RED instance with a shared router and a shared formatting subflow means a new app's alerts get the same visual treatment as every existing one automatically, and a fleet-wide change (say, adding a severity-based routing rule) happens in one place. The alternative — each app posting directly to a per-app Slack webhook, or a small script per integration — was rejected because it would mean N different message formats, N different places to add a new integration's logic, and no single point to add fleet-wide behavior (like the OS-level checks below) without touching every app individually. One Node-RED instance with a shared router and a shared formatting subflow means a new app's alerts get the same visual treatment as every existing one automatically, and a fleet-wide change (say, adding a severity-based routing rule) happens in one place.
> [!WARNING] > [!WARNING]
> **Real debt in the flow**: of the apps wired into the router, two no longer resolve to anything — NetBox was fully decommissioned (see [`docs/registry-and-build.md`](registry-and-build.md) for how stacks get retired) but its detection code was never swept out, and Kestra's alerts actually travel a completely different path than its own webhook route suggests: Kestra publishes its own alerts over MQTT, not HTTP, which makes its dedicated webhook route effectively dead code. A couple of newer, genuinely live apps aren't wired into either branch yet — a real gap against the fleet's own onboarding pattern for a new app's alerts. > **Real debt in the flow, one instance already cleaned up**: NetBox's dead detection code — left behind after the app was fully decommissioned (see [`docs/registry-and-build.md`](registry-and-build.md) for how stacks get retired) — has since been swept out of the flow entirely. What's still live: Kestra's alerts actually travel a completely different path than its own webhook route suggests — Kestra publishes its own alerts over MQTT, not HTTP, which makes its dedicated webhook route effectively dead code. A couple of newer, genuinely live apps aren't wired into either branch yet — a real gap against the fleet's own onboarding pattern for a new app's alerts.
This is the real routing table, not a claimed count — counted directly from the flow export, not restated from memory: This is the real routing table, not a claimed count — counted directly from the flow export, not restated from memory:
@@ -31,11 +31,11 @@ print('Switch rules:', len(switch['rules']))
dedicated = [n for n in flows if n.get('type')=='http in' and n.get('url','').endswith('-webhook') and n.get('url') != '/app-webhook'] dedicated = [n for n in flows if n.get('type')=='http in' and n.get('url','').endswith('-webhook') and n.get('url') != '/app-webhook']
print('Dedicated non-router endpoints:', len(dedicated)) print('Dedicated non-router endpoints:', len(dedicated))
" "
Switch rules: 16 Switch rules: 15
Dedicated non-router endpoints: 2 Dedicated non-router endpoints: 2
``` ```
16 rules = the 15 routed apps plus one `unknown` fallback that just drops the payload. And this is the real function behind the "completely different path" line above — the MQTT branch doesn't parse `msg.topic` looking for a known set of fleet hosts, it just takes whatever the second path segment is: 15 rules = the 14 routed apps plus one `unknown` fallback that just drops the payload. And this is the real function behind the "completely different path" line above — the MQTT branch doesn't parse `msg.topic` looking for a known set of fleet hosts, it just takes whatever the second path segment is:
``` ```
$ grep -A4 '"Inventory Parse Function' flows.json $ grep -A4 '"Inventory Parse Function' flows.json
@@ -48,9 +48,9 @@ return msg;
Topic-agnostic by construction — so a publisher that isn't a fleet host at all gets handled identically to one that is, which is exactly the mechanism that makes an app's *own* MQTT-published alerts real and its dedicated webhook route genuinely dead, not just theoretically bypassable. Topic-agnostic by construction — so a publisher that isn't a fleet host at all gets handled identically to one that is, which is exactly the mechanism that makes an app's *own* MQTT-published alerts real and its dedicated webhook route genuinely dead, not just theoretically bypassable.
This is the real flow, not a redrawn version of it — the App Router feeding a 16-way Switch, NetBox's and Kestra's mapper functions still sitting there wired in and unreachable, Kopia and Bazarr's dedicated endpoints bypassing all of it entirely on their way to the same shared subflow: This is the real flow, not a redrawn version of it — the App Router feeding a 15-way Switch, Kestra's mapper function still sitting there wired in and unreachable, Kopia and Bazarr's dedicated endpoints bypassing all of it entirely on their way to the same shared subflow:
![Real Node-RED canvas: HTTP-in webhook through App Router into a 16-rule Switch function feeding per-app mapper functions (including netbox mapper and kestra mapper, both still wired but dead), MQTT-in through the Inventory Parse Function, and Kopia/Bazarr's own dedicated HTTP-in nodes bypassing the router entirely — all converging on the shared MQTT → Slack subflow](../assets/notifications-node-red-flow.png) ![Real Node-RED canvas: HTTP-in webhook through App Router into a 15-rule Switch function feeding per-app mapper functions (including kestra mapper, still wired but dead since its real alerts travel over MQTT instead), MQTT-in through the Inventory Parse Function, and Kopia/Bazarr's own dedicated HTTP-in nodes bypassing the router entirely — all converging on the shared MQTT → Slack subflow](../assets/notifications-node-red-flow.png)
## Message Broker ## Message Broker
@@ -104,10 +104,10 @@ The Node-RED App Router figures out which app sent a webhook by inspecting the p
> >
> ``` > ```
> $ grep -oE '\$STATE_DIR/[a-zA-Z_]+' mqtt-monitor.sh.j2 | sed -E 's/(_alert|_last_count|_since|_report)$//' | sort -u | wc -l > $ grep -oE '\$STATE_DIR/[a-zA-Z_]+' mqtt-monitor.sh.j2 | sed -E 's/(_alert|_last_count|_since|_report)$//' | sort -u | wc -l
> 22 > 23
> ``` > ```
> >
> 22 from that count, plus disk-usage thresholds — which don't show up in that grep because its state key is built dynamically per mountpoint, not a fixed name — makes 23. One of those 22 was a genuine KB gap this doc caught mid-audit: crash-looping systemd units (`activating`/`auto-restart` stuck in a loop) never trips the plain failed-units check above it, because a unit that keeps getting restarted never actually reaches systemd's "failed" state. That distinction wasn't theoretical — it's in the script because of a real incident where exactly that happened and produced zero alerts. > Of that 23, one (`failed_units_current`) is a companion state file for the same "failed systemd units" check as `failed_units`, not a distinct 23rd check — so 22 real checks, plus disk-usage thresholds, which don't show up in that grep because its state key is built dynamically per mountpoint, not a fixed name, still makes 23 total. One of those 22 was a genuine KB gap this doc caught mid-audit: crash-looping systemd units (`activating`/`auto-restart` stuck in a loop) never trips the plain failed-units check above it, because a unit that keeps getting restarted never actually reaches systemd's "failed" state. That distinction wasn't theoretical — it's in the script because of a real incident where exactly that happened and produced zero alerts.
## One Formatter, Every Source ## One Formatter, Every Source