Skip to content
runlot
ObservabilitySettings

Settings

Sampling, caps and cost, log pipelines and the sensitive data scanner, OTLP and Logpush export, rehydration, tail consumers, the profiler, heap snapshots, roles and audit.

Everything here is under Observability → Settings, and each setting has an API route under /v1/orgs/{org}/projects/{project}/observability/.

Sampling

Request records are always kept. What sampling decides is whether an invocation's contents — its console lines and exceptions (logs), and its spans (traces) — are collected. Both are off by default, because building them costs the worker CPU on every request.

runlot.json
{
  "observability": {
    "logs": { "enabled": true, "head_sampling_rate": 0.25, "invocation_logs": true },
    "traces": { "enabled": true, "head_sampling_rate": 0.05 }
  }
}
KeyMeaning
logs.enabledCollect console lines and exceptions per invocation. Default false
logs.head_sampling_rateShare of invocations collected, greater than 0 and at most 1. Default 1
logs.invocation_logsfalse keeps exceptions but not console lines. Default true
traces.enabledCollect spans. Default false
traces.head_sampling_rateShare of invocations whose spans are kept. Default 1

The decision is made once per request, when it arrives, so a sampled request has all of its lines and spans and an unsampled one has none. What your worker prints to stdout and stderr is stored regardless.

The dashboard can override any of the four without a deploy. Each field you leave empty falls back to the live deployment's runlot.json:

curl -X PUT https://api.runlot.io/v1/orgs/my-org/projects/my-app/observability/sampling \
  -H "Authorization: Bearer $RUNLOT_TOKEN" -H "Content-Type: application/json" \
  -d '{"logsEnabled": true, "logSampleRate": 0.1, "tracesEnabled": null, "traceSampleRate": null}'

Requests pick up a change at the next routing-map refresh, usually within seconds and at most about 30 seconds later. Cron, queue, email and Durable Object alarm invocations pick it up the next time the app starts. Setting it needs the admin role.

Caps and cost

Every metered signal follows one rule:

  • The cap is on by default, at the plan's included amount.
  • Pro can raise it or remove it, per project. Usage past the included amount is billed, and the page shows the month's projected cost at the current rate before and after you change it.
  • Free is fixed at its included amount. Any plan can lower a cap.
  • The organization's spending limit is above every cap. Reaching it stops collection, never your app.
  • Reaching a cap degrades collection, never the record that it happened. Counts are always kept.
MeterIncluded, FreeIncluded, ProPast the capPro price past included
Log events200,000 a day per project2,000,000 a day per project1 % of events kept, weighted$0.60 per million
Spans200,000 a day per project2,000,000 a day per projectSuccessful detail at 1 %; errors kept$0.60 per million
Server error events5,000 a month100,000 a monthCounts kept, bodies sampled at 1 %$0.20 per 1,000
Custom metric series100 a month2,000 a monthNew series refused and counted$0.05 per series per month
RUM sessions—50,000 a monthNot recorded$0.50 per 1,000
Session replays—5,000 a monthNot recorded$3.00 per 1,000
Synthetic checks1 at 15 minutes20 at 1 minuteCannot add more$1.00 per check per month, pro rata
Profiler—Billed per profiled hour—$0.03 per hour
Long-kept traces—30 days—$0.10 per GB-month

Platform error events — the ones Runlot records when it had to answer for your app — are never capped.

curl -X PUT https://api.runlot.io/v1/orgs/my-org/projects/my-app/observability/caps \
  -H "Authorization: Bearer $RUNLOT_TOKEN" -H "Content-Type: application/json" \
  -d '{"logEventsDaily": 10000000, "customSeries": -1}'

Each of logEventsDaily, customSeries and rumSessions takes a number, -1 for no cap, or null for the plan's included amount; a meter you leave out is unchanged. Error and span caps are under runlot errors settings (--cap, --span-cap). Changing a cap needs the admin role and observability:admin.

Pipelines and the sensitive data scanner

Log events pass through a pipeline and a scanner at ingest, before they are stored and before they are exported. Keep both in .runlot/logs.json, which runlot deploy carries with the deployment:

.runlot/logs.json
{
  "pipeline": [
    { "type": "json_parse" },
    { "type": "regex", "pattern": "took (?P<took_ms>\\d+)ms" },
    { "type": "rename", "from": "uid", "to": "userId" },
    { "type": "level", "field": "severity", "mapping": { "W": "warn", "E": "error" } },
    { "type": "drop", "filter": [{ "key": "$message", "op": "starts_with", "value": "healthcheck" }] }
  ],
  "scanner": {
    "builtins": { "email": "hash", "card": "redact", "jwt": "redact" },
    "rules": [{ "name": "order-token", "pattern": "ot_[A-Za-z0-9]{24}", "action": "redact" }]
  },
  "logMetrics": []
}
  • Processors run in order: json_parse, kv, regex (RE2 with named groups), rename, level, url, user_agent, drop and sample. Up to 32. Without a pipeline only json_parse runs.
  • Built-in scanner rules: jwt, bearer, api_key, email, card (checked with Luhn) and kr_rrn (Korean resident registration numbers that pass their checksum). Each one is redact, hash, drop or off; a rule you do not mention redacts. Add up to 50 of your own RE2 rules. The same scanner runs over error events.
  • An event a pipeline or the scanner drops does not count toward the daily cap.
  • logMetrics is log-to-metrics.
  • The file is at most 64 KiB. It is applied when its deployment goes live; an edit made in the dashboard stands until the next deploy that carries the file. Export in the dashboard gives you the current configuration in this format.
  • Test a scanner against a sample string before you save it; nothing is stored.

Export

Send logs, traces and metrics to your own tools. Pro, up to 5 destinations per project.

  • OTLP: OTLP/HTTP JSON to an HTTPS collector on port 443 or 4318 at a public address, with optional headers (sealed; only their names are shown again). Signals logs, traces and metrics. With persist: false, logs and traces are exported without being stored by Runlot.
  • Logpush: hourly gzipped NDJSON objects of logs and invocations, written as <prefix>/yyyy/mm/dd/hh/<signal>-<n>.ndjson.gz, to an S3-compatible bucket you name or to the project's own env.storage bucket.

Each destination shows its delivery state and last failure. A slow or failing destination is retried with backoff and never holds up your app.

Rehydration

Logs older than the plan's retention can come back for querying from a Logpush archive. Pick a Logpush destination and a range of up to 72 hours; the part older than retention is loaded back and answers the same queries for 7 days. Rehydrated rows are not exported again and do not change the metric charts. Pro.

Tail consumers

A tail consumer is an export of your own worker that receives each invocation's trace from the runtime, as with Cloudflare's tail_consumers:

runlot.json
{ "main": "src/index.ts", "tail_consumers": [{ "entrypoint": "AuditTail" }] }
src/index.ts
import { WorkerEntrypoint } from "cloudflare:workers";

export class AuditTail extends WorkerEntrypoint {
  async tail(events) {
    for (const e of events) if (e.outcome !== "ok") console.log({ audit: e.outcome });
  }
}

export default { fetch: () => new Response("ok") };

The entrypoint must be an export of the main module, not a Durable Object class and not a name starting with runlot. Every invocation builds a trace for it whether or not it reads it, so it costs CPU. Free allows 1 and Pro 5.

Profiler and heap snapshots

The continuous profiler samples a CPU profile of your app for 10 seconds every 10 minutes or so, symbolicates it with the deployment's source maps, and shows a flame graph per version, the top functions by self time, and a comparison between two versions. Profiles are linked from slow traces of the same minute.

It is Pro and off by default. Turning it on takes effect at the next deployment or restart; nothing restarts your app to attach it. Profiled time is billed at $0.03 an hour.

With the profiler on, an admin can take a heap snapshot of the running app. The app pauses while it is taken, and the snapshot holds everything the app holds in memory, secrets included — so it is admin only and audited, one at a time and at most one every 10 minutes, and it expires after 7 days.

Roles and audit

Three permissions govern observability data on a project:

PermissionAllows
observability:readMetrics, traces, logs
observability:piiUser identifiers, IP addresses where stored, replays, unscrubbed fields
observability:adminPipelines, scanners, monitors, caps, exports, retention, roles

Organization viewers and members get read; admins get all three. An admin can widen or narrow them for one person on one project; pii and admin need read beside them, and you cannot remove your own admin. A personal access token carries only the scopes it was created with, so a token narrows them further.

The audit trail records every configuration change — caps, exports, rehydration, role overrides, pipelines, scanners, monitors — and every read of PII-scoped data. If you send a bug report, reads of it by Runlot staff appear there too. Reading the trail needs observability:admin.

Data lists where the project's observability data is kept and where it is exported to.

On this page