Dead man's switch monitoring

Alert when the ping doesn't arrive

Normal monitoring probes your service and alerts on failure. A dead man's switch inverts that: the thing being monitored sends a periodic heartbeat, and the monitor alerts when the heartbeat stops. It's the only pattern that catches failures where nothing is left running to report an error:

• a nightly backup or report job that never started
• a monitoring agent inside a private network that crashed
• a Prometheus + Alertmanager pipeline that is up but silently not delivering alerts

In Wanepia this is a push check. Once it has received at least one result, the scheduler continuously watches it. When no result arrives within the grace window (now − last_result_time > grace), Wanepia records a synthetic failed result, re-aggregates the entity's status, and fires your configured notifications — Slack, Discord, Telegram, ntfy, webhook, or NATS — exactly like a real probe failure. When the heartbeat resumes, the entity recovers automatically.

Heartbeat check in two curl calls

1Create a push check

Attach it to any entity in your catalog. Set failure_threshold: 1 — heartbeats are infrequent, so one missed beat should alert immediately.

curl -X POST https://api.wanepia.com/v1/checks \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_id": "<entity-id>",
    "check_type": "push",
    "execution_mode": "push",
    "name": "Nightly backup heartbeat",
    "interval_seconds": 60,
    "failure_threshold": 1,
    "config": { "grace_seconds": 180 }
  }'

The response includes the check's id. The grace window defaults to interval_seconds × 2 — one missed beat plus jitter — and config.grace_seconds overrides it per check.

2Send heartbeats

From your own agent or cron job, post real results with latency and error detail:

curl -s -X POST "https://api.wanepia.com/v1/checks/$CHECK_ID/results" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "success": true,
    "latency_ms": 45,
    "error_message": ""
  }'

Or, for third-party systems whose payload you don't control, use the heartbeat endpoint — it ignores the request body entirely and records any authenticated POST as a successful heartbeat:

curl -s -X POST "https://api.wanepia.com/v1/checks/$CHECK_ID/heartbeat" \
  -H "Authorization: Bearer $TOKEN"

That's the whole switch. Stop pinging, get paged.

Deadman switch for Prometheus Alertmanager

Prometheus setups conventionally define a DeadMansSwitch (or Watchdog) alert with expr: vector(1) that is always firing. If it ever stops arriving at its receiver, the alerting pipeline itself is broken — Prometheus is down, Alertmanager is down, or the network path between them is. Point that alert at a Wanepia heartbeat endpoint to close the loop with a system outside your own stack:

route:
  routes:
    - match:
        alertname: DeadMansSwitch
      receiver: wanepia-deadman
      repeat_interval: 1m

receivers:
  - name: wanepia-deadman
    webhook_configs:
      - url: https://api.wanepia.com/v1/checks/<check-id>/heartbeat
        http_config:
          authorization:
            type: Bearer
            credentials: <your-api-key>
        send_resolved: false

Create the push check with interval_seconds matching your repeat_interval and failure_threshold: 1. While Prometheus, Alertmanager, and the network path are healthy, the heartbeat keeps arriving and the entity stays up. If any of them dies, the grace window expires and Wanepia alerts you through an independent channel that your monitoring itself is down.

Cron and private-network agents

The same push check monitors anything on a schedule. This script probes a Postgres instance behind a firewall and reports the result — run it from cron or a Kubernetes CronJob on any host that can reach the database:

#!/usr/bin/env bash
set -euo pipefail

start=$(date +%s%3N)

if pg_isready -h "$DB_HOST" -p "$DB_PORT" -q; then
  payload='{"success":true,"latency_ms":'$(( $(date +%s%3N) - start ))',"error_message":""}'
else
  payload='{"success":false,"latency_ms":'$(( $(date +%s%3N) - start ))',"error_message":"pg_isready: host unreachable or refusing connections"}'
fi

curl -s -X POST "https://api.wanepia.com/v1/checks/$CHECK_ID/results" \
  -H "Authorization: Bearer $WANEPIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$payload"

* * * * * /usr/local/bin/wanepia-pg-check.sh

This covers both failure modes at once: if pg_isready fails you get an alert with the error message; if the host running cron dies, the heartbeat stops and the grace window catches it. The dashboard distinguishes the two — stale push checks show a warning banner, so you can tell "the probe ran and failed" from "the probe stopped running entirely". Details in the docs under push checks.