diff --git a/assets/observability-log-fanout.svg b/assets/observability-log-fanout.svg index 382f26c..05254b6 100644 --- a/assets/observability-log-fanout.svg +++ b/assets/observability-log-fanout.svg @@ -1,4 +1,4 @@ -
journald
system logs
Loki
job=system
Docker API
container logs, survives recreation
Loki
job=docker
DNS query log
daily rotation
Loki
job=dns, real event time
 \ No newline at end of file +
journald
system logs
Loki
job=system
Docker API
container logs, survives recreation
Loki
job=docker
DNS query log
daily rotation
Loki
job=technitium-dns, real event time
auditd log
6 hosts, gap-fill
Loki
job=auditd, real event time
 \ No newline at end of file diff --git a/docs/observability.md b/docs/observability.md index 5b52486..32b2fbe 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -1,5 +1,5 @@ - @@ -48,7 +48,9 @@ SystemKeepFree=1G MaxRetentionSec=14day ``` -![Three independent log fan-outs, each landing in Loki under its own job label: journald as job=system, Docker container logs (via the container API, not path-tailing) as job=docker, and the DNS query log as job=dns with real event time instead of ingestion time](../assets/observability-log-fanout.svg) +![Four independent log fan-outs, each landing in Loki under its own job label: journald as job=system, Docker container logs (via the container API, not path-tailing) as job=docker, the DNS query log as job=technitium-dns with real event time instead of ingestion time, and auditd's own log as job=auditd on the 6 hosts that carry it](../assets/observability-log-fanout.svg) + +A fourth, newer branch ships raw auditd logs to Loki too, on the 6 hosts that carry the `auditd_hardening` compliance role (automation, caddy, monitoring, technitium, vault, wazuh) — a real gap-fill after one host's own `/var/log/audit` once filled to 100% with zero external visibility to catch it. It doesn't replace the local cleanup timer that actually bounds that disk usage; it just means the data isn't only ever local anymore. This is real log data flowing through that pipeline, not a mockup — `job=technitium-dns` entries showing real DNS query/answer content, `job=docker` entries from real containers, and `job=system` from journald itself, all landing in the same Loki instance: