runlot docs

Monitors that alert by email, Slack, webhook, PagerDuty or Opsgenie, SLOs with burn-rate alerts, downtimes, synthetic checks, and monitors kept in your repository.

Monitors

A monitor evaluates a query every minute and notifies you when it crosses a threshold. Create one under Observability → Monitors, through the API, or as a file in your repository.

Type	Watches
`apm`	A route's `requests`, `errors`, `error_rate`, `status_4xx`, `status_5xx`, or latency (`latency_avg`, `latency_p50` … `latency_p999`)
`database`	An `env.db` fingerprint's `calls`, `errors`, latency, or `wait_share`
`infra`	`memory_pct`, `memory_bytes`, pressure stall, `oom_kills`, `oom_exits`, `high_events`, `cpu_throttled_us`
`metric`	A custom metric, with an aggregation
`change`	How much a metric moved over the last N minutes, in absolute terms or percent
`anomaly`	A deviation from the same window on each of the previous 14 days (needs 3 days of history)
`log`	A count of log events matching a level, source, event type or text
`error`, `issue`	Error events, or a Runlot Errors issue rule (new, regressed, escalating)
`cron`	A schedule that did not run or failed
`uptime`	A synthetic check
`slo_burn`	An SLO's error budget burn rate
`composite`	A boolean of other monitors: `a && (b

Each monitor has a window (5 minutes by default), a frequency (1 minute), an evaluation delay for late data (60 seconds), alert and warn thresholds, and optional recovery thresholds so a value hovering at the line does not flap. groupBy sends one alert per group — per route, say — instead of one for everything. noData decides what an empty window means: ignore it (the default), alert, or resolve. A count with no rows is zero, so a route that stops failing recovers.

The message is a template: {{value}}, {{threshold}}, {{state}}, {{monitor.name}}, {{project}}, {{link}}, and each group dimension such as {{route}}.

Free has 10 monitors per project and Pro 500.

Destinations

Monitors notify the project's alert destinations, the same ones Runlot Errors uses: email, Slack, webhook, PagerDuty and Opsgenie. A Slack or webhook URL, a PagerDuty routing key or an Opsgenie API key (prefix eu: for the EU instance) is sealed when you save it and never shown again. An external destination stays off until you enable it and acknowledge that it receives alert data. PagerDuty incidents resolve and Opsgenie alerts close on their own when the monitor recovers.

Send a test notification from the monitor's page before you rely on it.

Downtimes

A downtime mutes monitors by ID or by tag, once or on a daily or weekly schedule. A monitor can also mute itself for N minutes after each deploy (muteAfterDeployMinutes). Muting defers, it does not drop: if a group is still alerting when the downtime ends, the alert goes out then. A recovery of an alert that was already sent is never muted.

SLOs

An SLO is a target over 7, 30 or 90 days.

Request-based: a request is good when it is not a 5xx and finished within a latency target. The target is one of 50, 100, 200, 300, 500, 750, 1000, 1500, 2000, 3000, 5000 or 10000 ms, and the count is exact, not estimated. Scope it to up to 50 routes.
Monitor-based: the share of time a monitor spent with every group OK.

Each SLO shows the remaining error budget and a burn chart. A slo_burn monitor alerts on burn rate with two windows at once, so a fast burn pages and a slow one does not wake anyone: by default 1 hour and 5 minutes at 14.4× is an alert, and 6 hours and 30 minutes at 6× is a warning.

Free has 1 SLO per project and Pro 50. A request-based SLO counts requests from the moment it is created.

Synthetic checks

A synthetic check calls one of your project's own HTTPS addresses from the control plane, on a schedule, and checks the answer. It wakes a parked app, which is the point.

runlot errors uptime add https://my-app.me.runlot.app/health 5
runlot errors uptime list
runlot errors uptime runs <id>

Every 1, 5 or 15 minutes. Free: 1 check, at most every 15 minutes. Pro: 20 checks, every minute.
Assertions on the status, a header, the body, a JSON path ($.status) or the latency. Redirects are not followed; a 3xx is the answer.
A multi-step check runs up to 10 requests in order and carries values forward: extract a token from a JSON path or a header in one step and use it as {{token}} in the next. Headers marked secret are sealed.
Two failures in a row open one issue; the next success resolves it. An uptime monitor turns the result into an alert, and a monitor-based SLO into an uptime target.
Check traffic is tagged synthetic and left out of APM numbers by default.

On Pro, check runs past the included 20 checks at a 1-minute interval are billed at $1.00 per check per month, pro rata.

Monitors as code

Keep monitors in your repository under .runlot/monitors/. Every .json file there holds one monitor or an array of them, and they are applied when the deployment goes live:

.runlot/monitors/checkout-5xx.json

{
  "name": "Checkout 5xx",
  "type": "apm",
  "query": { "metric": "error_rate", "filter": { "route": "/checkout" } },
  "options": {
    "windowMinutes": 5,
    "thresholds": { "alert": 5, "warn": 2 },
    "groupBy": ["route"]
  },
  "message": "Checkout errors at {{value}}% on {{route}}. {{link}}",
  "destinations": ["on-call"]
}

A lone monitor without a key takes the file's name (checkout-5xx). In an array, each needs its own key. The key is how the next deploy finds the same monitor.
Destinations are named, not by ID, so the file has no secrets in it.
An empty .runlot/monitors/ directory removes every monitor that came from the repository. No directory changes nothing.
A monitor from the repository is read-only in the dashboard; change the file and deploy. Preview deployments never apply monitors.
Export on the Monitors page, or GET …/observability/monitors/export, gives you the monitors made in the dashboard in this file format, ready to commit.

Monitors and SLOs