docs: add orchestration, registry, dashy, cron, and content pipeline write-ups
Completes the orchestration/registry/content batch - Kestra's check-pause-approve-apply pattern, the self-hosted build pipeline, the SOC dashboard's 14-source aggregation, the cron heatmap, and the in-progress content pipeline, all in the diagram+callout+table style.
This commit is contained in:
5 files changed
+248
No files matched your search
@@ -0,0 +1,32 @@
|
||||
# ⏱️ Cron Scheduler
|
||||
|
||||
> [!NOTE]
|
||||
> A fleet-wide inventory of every known scheduled job — orchestration flows, system timers, automation-managed cron, app-internal schedules — visualized as a heatmap so a new job's time can be picked without colliding with existing load.
|
||||
|
||||
## The heatmap
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Sources["Job sources<br/>flows · timers · cron · app schedules"] --> Table["Structured inventory<br/>name, host, category,<br/>schedule, duration, resource tags"]
|
||||
Table --> Weight["Weighted load calculation"]
|
||||
Weight --> Grid["7×24 grid<br/>day × hour"]
|
||||
Grid --> Bands["4 color bands<br/>quieter → busier"]
|
||||
```
|
||||
|
||||
Each cell's color reflects a **computed load weight**, not a raw job count: a base weight per job, plus a duration-based weight, plus resource-intensity weight (disk/CPU/network, summed if a job carries multiple tags), plus an extra weight for anything tagged as a backup job. Jobs whose time can't be resolved to a specific day and hour are excluded from the grid and listed separately rather than forced into a misleading cell.
|
||||
|
||||
> [!TIP]
|
||||
> Clicking a populated cell expands the full job list for that slot — name, host, category, and whether that job's schedule can actually be moved or is fixed by whatever app owns it.
|
||||
|
||||
## Honest about its own limitation
|
||||
|
||||
> [!WARNING]
|
||||
> **This is a manually-maintained inventory, not an auto-refreshed one.** The schema has a "last synced" column, but nothing currently populates it — no automated process re-scans the fleet to keep this current. Entries get added or edited by hand as jobs are created or changed. Documented as a real limitation rather than presented as more automated than it actually is.
|
||||
|
||||
## A real, unresolved bug this system surfaced
|
||||
|
||||
> [!IMPORTANT]
|
||||
> One backup job's prune step targets a datastore name that doesn't actually exist — not a scheduling collision, an orphaned config that's been silently failing. Two fixes are on the table (drop the dead job and let a different retention mechanism be authoritative, or repoint it at the real datastore name and let its richer retention rules take over) — deliberately not resolved yet, tracked openly rather than silently patched over with a guess.
|
||||
|
||||
---
|
||||
Related stacks: [`stacks/platform/dashy/`](../stacks/platform/dashy/), [`stacks/platform/nocodb/`](../stacks/platform/nocodb/)
|
||||
Reference in new issue
Block a user