---
name: runlot-ai
description: Call language and embedding models from a Runlot worker with env.ai — env.ai.run options (messages, system, reasoning effort, responseFormat JSON Schema, prompt cache, streaming), server-side tools and toolMode "return" for your own tool loop, embeddings with env.vectors, usage and usage.unreported, credit, limits and errors, team subscription lending (BYOS), and agent frameworks (Vercel AI SDK, Mastra, OpenAI Agents SDK, LangGraph) through @runlot/ai-sdk-provider. Use when adding AI features, chat, agents, RAG or structured extraction to a Runlot app, or when an env.ai call fails or costs more than expected.
---

# AI on Runlot

Every project's worker has `env.ai`. The binding is the credential: there is no API key, no
external endpoint, and no CLI command that calls a model. Calls are billed to the org's credit
(frontier models at the provider's list price) or a small free daily allowance for value models.

Docs: https://docs.runlot.io/docs/data/ai/index.mdx

## Pick a model from the catalog, not from memory

```bash
runlot ai models            # ids, kind (chat/embedding), prices per 1M tokens
runlot ai models --json     # effortLevels, structuredOutput, cacheTTLs per model
```

In code, `await env.ai.models()` returns the same rows. Model ids look like
`anthropic/claude-sonnet-5`, `openai/gpt-6-luna`, `google/…` (frontier, need credit) and
`@runlot/nova-micro` (value tier, covered by the $0.11/day free allowance). An id that is not in
the catalog is `400 unknown_model`.

## Call it

```ts
const r = await env.ai.run("anthropic/claude-sonnet-5", {
  system: "You write order emails.",                 // the system prompt goes here, not in messages
  messages: [{ role: "user", content: "Write the confirmation for order A-123." }],
  maxTokens: 1024,
});
r.text; r.stopReason; r.usage; r.usageComplete; r.model;
```

`text` can be empty on success — a reasoning model can spend `maxTokens` on reasoning and stop
with `stopReason: "max_tokens"`. Log `stopReason`, `usage` and `usageComplete` together.

| Need | Option | Reference |
|---|---|---|
| JSON in a fixed shape | `responseFormat: { type: "json_schema", schema }` | [references/run.md](references/run.md) |
| Think harder or faster | `reasoning: { effort }` | [references/run.md](references/run.md) |
| Long shared prefix, many calls | `cache`, `cacheBreakpoint` | [references/run.md](references/run.md) |
| Stream to the browser | `stream: true` → `ReadableStream` of SSE | [references/run.md](references/run.md) |
| Model calls your functions | `tools: [{ name, description, inputJsonSchema, execute }]` | [references/tools.md](references/tools.md) |
| Your loop or a framework runs the tools | `tools` without `execute` + `toolMode: "return"` | [references/tools.md](references/tools.md) |
| AI SDK, Mastra, OpenAI Agents, LangGraph | `@runlot/ai-sdk-provider` | [references/agent-frameworks.md](references/agent-frameworks.md) |
| Embeddings and search | `env.ai.embed`, `env.vectors` | [references/embeddings.md](references/embeddings.md) |
| Costs, usage, limits, errors, BYOS | — | [references/usage-and-errors.md](references/usage-and-errors.md) |

## Check spend and limits

```bash
runlot ai                   # calls, tokens, spend and limits for this project
runlot ai credit            # the org's balance and automatic top-up
```

The dashboard's **AI** section lists every call (model, lane, tokens, price, request link).
Pass `metadata: { sessionId, userId, tags }` on calls to group them there.

## Do

- Choose ids and capabilities from `runlot ai models` / `env.ai.models()`.
- Set `maxTokens`; on Free it is capped at 4,096 per call.
- Handle `429` (`rate_limited`, `too_many_concurrent`) and `402 no_credit` explicitly.
- Validate model output (`JSON.parse` + schema check) before acting on it.
- Treat a tool with side effects as possibly executed when a run throws; check your own records before retrying.

## Don't

- Don't put `role: "system"` in `messages`; roles are `user` and `assistant` only.
- Don't `JSON.parse` the text of a tool run with `responseFormat` before checking `r.formatted`.
- Don't add a provider SDK with your own API key; there is no bring-your-own-key path.
- Don't run long agent loops inside one request on Free (2 concurrent calls, 30 per minute per project); move them to a queue consumer.
- Don't call `env.ai` at module top level; call it inside a handler.