alerting
Deciding what is worth waking someone for, and proving the alerting itself is alive.
-
Turning Alertmanager Webhooks Into Retained MQTT State
Alertmanager webhook to MQTT retained messages: clear stale state on resolve with an empty retained payload, keeping current alerts in Redis by fingerprint.
-
A Dead-Man's Switch for the Monitoring System Itself
Alertmanager dead man's switch with expr: vector(1) == 1: an always-firing Heartbeat alert and an external receiver that alerts when pings stop.
-
Alerting on Symptoms Instead of Metrics: Designing Alerts People Do Not Ignore
probe_success == 0 via blackbox_exporter instead of CPU alerts, plus a dnspython check that treats NOERROR with an empty answer as down.