diff --git a/assets/security-monitoring-sca-dashboard.png b/assets/security-monitoring-sca-dashboard.png new file mode 100644 index 0000000..3a8cece Binary files /dev/null and b/assets/security-monitoring-sca-dashboard.png differ diff --git a/docs/security-monitoring.md b/docs/security-monitoring.md index 06bb603..ee75ba9 100644 --- a/docs/security-monitoring.md +++ b/docs/security-monitoring.md @@ -39,7 +39,17 @@ The alternative to an agent on every host is a purely network-level monitoring a ## A fleet-wide bug this system caught in itself > [!IMPORTANT] -> Wazuh's own default agent log-queue buffer (1024 entries) overflowed fleet-wide during Ansible orchestration bursts — enough hosts logging enough events at once that the queue filled before the agent could forward them, silently dropping data rather than erroring loudly. Found, not assumed: root-caused to the default queue size specifically, then fixed with a dedicated tuning role that raises `logcollector.queue_size` on every agent. That role is deliberately narrow — it only touches queue size and enabled/started state; syscheck ignores, rootcheck ignores, and other per-agent tuning are still one-off manual edits, honestly left uncovered rather than implied as handled by a role that doesn't actually reach them. +> Wazuh's own default agent log-queue buffer (1024 entries) overflowed fleet-wide during Ansible orchestration bursts — enough hosts logging enough events at once that the queue filled before the agent could forward them, silently dropping data rather than erroring loudly. Found, not assumed: root-caused to the default queue size specifically, then fixed with a dedicated tuning role that raises `logcollector.queue_size` on every agent. That role is deliberately narrow — it only touches queue size and enabled/started state; syscheck ignores, rootcheck ignores, and other per-agent tuning are still one-off manual edits, honestly left uncovered rather than implied as handled by a role that doesn't actually reach them. This is the real root-cause comment from the fix itself, not a summary of it: +> +> ``` +> $ cat wazuh_agent_tuning/tasks/main.yml +> # Root-caused 2026-07-28: the default 1024-entry logcollector->agentd queue +> # overflows during bursts of heavy journal activity (ansible fan-out runs +> # touching many hosts at once, container restarts, etc.) - Wazuh's own +> # "Agent event queue is full/90% full" rules (202/203) fired fleet-wide +> # during that night's role redeploys, confirmed via the indexer across +> # 9 of the fleet's agents. +> ``` ## How an alert gets handled @@ -94,7 +104,9 @@ One rule-writing gotcha that's easy to get backwards: a rule's `` fi ![A CIS-oriented hardening framework of 9 tagged Ansible sub-roles applies cleanly to most hosts and gets enforced fleet-wide, re-applied every compliance run — but two controls are deliberately skipped with the reason documented: kernel forwarding flipped for Docker hosts since container networking needs it, and SSH TCP forwarding kept permissive fleet-wide since real remote access depends on it](../assets/security-monitoring-compliance-gate.svg) -This is the same tag-driven selection model described in [`docs/iac-pipeline.md`](iac-pipeline.md) — which controls apply to a given host is explicit in that host's own tag list, not an implicit "all or nothing" blanket. SCA scores below 100% on any given host are an accepted tradeoff, not tracked as open bugs to close one by one — the hardening rollout itself is the actual control, not a perfect per-control score. Two real deviations from the strict CIS baseline exist fleet-wide, and both are documented with their actual reasoning rather than left as unexplained exceptions — neither is a gap waiting to be closed, both are permanent, reasoned trade-offs. +This is the same tag-driven selection model described in [`docs/iac-pipeline.md`](iac-pipeline.md) — which controls apply to a given host is explicit in that host's own tag list, not an implicit "all or nothing" blanket. SCA scores below 100% on any given host are an accepted tradeoff, not tracked as open bugs to close one by one — the hardening rollout itself is the actual control, not a perfect per-control score. Two real deviations from the strict CIS baseline exist fleet-wide, and both are documented with their actual reasoning rather than left as unexplained exceptions — neither is a gap waiting to be closed, both are permanent, reasoned trade-offs. This is a real score, not a hypothetical one — one host's actual compliance dashboard: + +![A real Wazuh endpoint dashboard: CIS Rocky Linux 10 Benchmark compliance at 87% (139 passed, 19 failed, 8 not applicable) — below 100%, exactly the accepted-tradeoff case described above, not a perfect score being claimed](../assets/security-monitoring-sca-dashboard.png) The authentication stack is configured to fail **safe** under a specific failure mode rather than fail open — a subtler design choice than it sounds, and the kind of detail that's easy to get backwards under pressure while writing the config.