Providers

@voltro/ai exposes one surface behind a provider abstraction. Pick yours via AI_PROVIDER. The same free functions — generateText({ prompt }), generateObject({ prompt, schema }), streamText({ prompt }) — work against any provider.

There is no ctx.ai. AI lives in the free functions you import from @voltro/ai, not on the request context.

#Supported providers

Provider	AI_PROVIDER value	Default model	Key env var	Notes
Anthropic	anthropic	claude-opus-4-8	ANTHROPIC_API_KEY	Claude. Best for agentic tool use + long context.
OpenAI	openai	gpt-5.5	OPENAI_API_KEY	GPT models via @ai-sdk/openai.
Gateway	gateway	none (required)	AI_GATEWAY_API_KEY	The Vercel AI Gateway: ANY model the AI SDK can reach through ONE key, addressed by a creator/model id (openai/gpt-5.5, anthropic/claude-opus-4-8, google/gemini-2.5-pro). No per-vendor @ai-sdk/* package needed.
Mock	mock	mock	none	Deterministic responses for tests + CI. Echoes the prompt (or scripted output).

mock is the default — voltro dev runs key-free out of the box, and every smoke test stays deterministic without a network call. Set AI_PROVIDER=anthropic / openai / gateway to use a real model.

Direct provider vs gateway: reach for a direct provider (anthropic / openai) when you want that vendor's own key + native behaviour. Reach for gateway when you want "any model, one key" without adding a new @ai-sdk/* package — the model id carries the vendor (openai/gpt-5.5). All four packages stay pinned to the same @ai-sdk/provider major; mixing a newer-major provider package in would break the shared model type.

#Setting the provider

AI_PROVIDER=anthropic AI_MODEL=claude-opus-4-8 ANTHROPIC_API_KEY=sk-ant-… voltro dev

Env vars providerFromEnv() reads:

Var	Default	Notes
AI_PROVIDER	mock	mock | anthropic | openai | gateway.
AI_MODEL	per provider (see below)	Override the model. Defaults: claude-opus-4-8 (anthropic), gpt-5.5 (openai), mock (mock). REQUIRED for gateway — a creator/model id; boot throws if unset.

Provider API keys are read by the underlying @ai-sdk/* packages from their standard env vars — ANTHROPIC_API_KEY, OPENAI_API_KEY, AI_GATEWAY_API_KEY. The framework doesn't read a separate AI_API_KEY. (To give a SINGLE agent its own key from code instead of env, see Per-config key + base URL below.)

#Anthropic

AI_PROVIDER=anthropic
AI_MODEL=claude-opus-4-8
ANTHROPIC_API_KEY=sk-ant-…

Anthropic-specific:

• Prompt caching — automatic for long, repeated prefixes via the AI SDK.
• Tool calling — Claude's native tool format; the framework's defineTool(...) adapts to it.

For long contexts Claude is the practical pick — fewer "context overflow" surprises.

#OpenAI

AI_PROVIDER=openai
AI_MODEL=gpt-5.5
OPENAI_API_KEY=sk-…

Backed by @ai-sdk/openai. AI_MODEL is a plain OpenAI model id (gpt-5.5, gpt-4o-mini, …). For a self-hosted / Azure-style / proxy endpoint, set baseURL on a ProviderConfig (see Per-config key + base URL) rather than an env var.

#Vercel AI Gateway

AI_PROVIDER=gateway
AI_MODEL=openai/gpt-5.5            # creator/model id — REQUIRED, no default
AI_GATEWAY_API_KEY=…

The gateway reaches EVERY model the AI SDK can address through ONE key — you never add a per-vendor @ai-sdk/* package. The model id encodes the vendor: openai/gpt-5.5, anthropic/claude-opus-4-8, google/gemini-2.5-pro. There's no sensible default model (the id IS the choice), so AI_PROVIDER=gateway with no AI_MODEL throws at boot with a pointer to fix it.

Use the gateway when you want to switch models across vendors freely from config; use a direct provider when you want that vendor's own key + native quirks (Anthropic prompt-caching, etc.).

#Live model catalog — getAvailableModels

A model-picker UI shouldn't hard-code a model list that goes stale. getAvailableModels() lists the gateway's models live — each with its modality + pricing — so the picker is always current:

import { getAvailableModels } from '@voltro/ai'

// Reads AI_GATEWAY_API_KEY; pass { apiKey, baseURL } to target a specific gateway.
const models = await getAvailableModels({ modality: 'language' })   // filter optional
// → GatewayModelInfo[]: { id, name, description?, modality, pricing? }
//   id        — the creator/model id you pass as the model ('openai/gpt-5.5')
//   modality  — 'language' | 'embedding' | 'image' | 'unknown'
//   pricing   — { inputPer1M, outputPer1M, cachedInputPer1M? } (USD per 1M tokens)

Pricing is normalised to USD per 1,000,000 tokens, the same shape as the cost toolkit's ModelPrice — so a catalog entry can feed the cost ledger directly (estimateCostUsd(usage, { model, price })). getAvailableModels is async (a plain Promise, not an Effect) and gatewayProvider is injectable for tests, so a picker query can call it without a live gateway in CI.

#Per-config key + base URL

Every ProviderConfig accepts an optional apiKey and baseURL. When EITHER is set, the framework builds a dedicated provider instance from it (createOpenAI({ apiKey }) / createAnthropic(...) / createGateway(...)) instead of the env-default singleton. Omit both → it falls back to the standard env var. The mock provider ignores both.

// A one-off call against a specific key + endpoint:
const r = yield* generateText({
  prompt,
  provider: { name: 'openai', model: 'gpt-5.5', apiKey: process.env.TEAM_OPENAI_KEY, baseURL: 'https://my-proxy/v1' },
})

This is the mechanism behind per-agent keys and BYOK (bring-your-own-key): a defineAgentExecutor can carry a static model: { name, model, apiKey }, OR a model: (input) => ({ …, apiKey: input.apiKey }) function that derives the key from the request. The key stays server-side (the executor file never reaches the browser) and is NOT persisted — only the prompt is stored in agent_messages. The full pattern (dynamic model picker, cost/tier routing, BYOK, and the security footguns) lives in Agents → Per-agent model + key.

#Mock (for tests)

AI_PROVIDER=mock

Every call returns deterministic output (it echoes the prompt, or a scripted turn sequence). Useful for:

• CI runs where you don't want real API calls
• Unit tests of agents — assert on the call shape, not the output
• Local dev when you're offline

Configure the mock per-test. useMockAi installs a process-global mock provider that every generateText / generateObject / streamText / runAssistant call resolves to (overriding AI_PROVIDER); call reset() to restore:

import { useMockAi } from '@voltro/ai/test'

let mock: ReturnType<typeof useMockAi>
beforeEach(() => {
  mock = useMockAi({
    // canned generateText / generateObject text
    generate: { text: 'Mocked summary text.' },
    // streamText token sequence (drives runAssistant deltas)
    stream:   ['Hello, ', 'world.'],
    // scripted tool-call → text turns for the tool loop (one per step)
    turns: [
      { toolCalls: [{ name: 'searchDocs', input: { query: 'x' } }] },
      { text: 'Based on the docs.' },
    ],
  })
})
afterEach(() => { mock.reset() })

mockAi({...}) returns a MockAi value with the same fixture shape — handy as a test helper for code that wants a mock to assert against.

throwNTimes(n, value) is a helper for retry tests — a function that throws the first n calls, then returns value.

#The call surface

One-shot text:

import { generateText } from '@voltro/ai'
import { Effect } from 'effect'

export default (input: { prompt: string }) =>
  Effect.gen(function* () {
    const { text, usage } = yield* generateText({ prompt: input.prompt })
    return { text }
  })

GenerateTextOptions is { prompt, system?, provider?, fallbacks?, maxTokens? }. There is no messages/effort shape — the prompt is a single string the SDK wraps as the user turn; system steers it. fallbacks is a provider fallback chain.

Structured output:

import { generateObject } from '@voltro/ai'
import { Schema } from 'effect'

const Summary = Schema.Struct({ title: Schema.String, bullets: Schema.Array(Schema.String) })

const { object } = yield* generateObject({ prompt: input.text, schema: Summary })

#Switching providers per call

Pass provider to override the env default for one call:

import { generateText } from '@voltro/ai'

const r = yield* generateText({
  prompt,
  system,
  provider: { name: 'anthropic', model: 'claude-opus-4-8' },
})

provider is a ProviderConfig — a discriminated union on name, so each provider only accepts the fields that apply to it:

type ProviderConfig =
  | { name: 'mock';      model?: string; mockText?: string; script?: MockScript }  // mock-only fields
  | { name: 'anthropic'; model: AnthropicModel; apiKey?: string; baseURL?: string }
  | { name: 'openai';    model: OpenAIModel;    apiKey?: string; baseURL?: string }
  | { name: 'gateway';   model: `${string}/${string}`; apiKey?: string; baseURL?: string }  // creator/model

The mock-only mockText / script can't appear on a real provider (the type rejects it), the gateway's model is a creator/model-typed string (a bare 'gpt-5.5' is a compile error, not a boot crash), and the per-provider model-id types (AnthropicModel / OpenAIModel) are open unions — known ids autocomplete, but any string the provider ships tomorrow still type-checks. Use provider for per-request model selection (e.g. a cheaper model on a fallback path), or to pin a specific key/endpoint (see Per-config key + base URL).

For runtime provider switching across a whole layer, bind an AiServiceImpl to the AiService Context tag at boot and read it with yield* AiService — defaultAiService (backed by providerFromEnv) is the default.

#Fallback chain — survive a provider outage

Pass an ordered fallbacks list to fall through to another provider/model when the primary FAILS. When the primary call fails — after its own retries — the call re-runs against fallbacks[0], then fallbacks[1], … until one succeeds; exhausting every option surfaces the LAST provider's typed error. So an Anthropic outage transparently drains to OpenAI (or the gateway) when a key is configured:

import { generateText } from '@voltro/ai'

const r = yield* generateText({
  prompt,
  provider:  { name: 'anthropic', model: 'claude-opus-4-8' },
  fallbacks: [
    { name: 'openai',  model: 'gpt-5.5' },
    { name: 'gateway', model: 'google/gemini-2.5-pro' },
  ],
})

• Applies to generateText, generateObject, generateObjectWithTools, and the stream surface. For streaming, use streamTextWithFallback(options) or streamTextWithRetry(options, retry) — the latter retries each provider maxAttempts times BEFORE moving to the next. A plain streamText uses only the primary.
• Only a provider (generation) failure falls through. A decode failure — the model answered but the output didn't satisfy the schema — surfaces immediately, because another provider won't fix a schema/prompt problem.
• Streams fall through only before content flows. Once any token/tool event has streamed the answer is committed and a later error surfaces as-is (re-running elsewhere would duplicate output); a deliberate cancelled never triggers a fallback.
• No fallbacks ⇒ unchanged single-provider behavior.

Typed errors are preserved end-to-end: a fully-exhausted chain fails with the last AiError on the Effect channel (generate) or a terminal error event (stream). Cost/usage is attributed to whichever provider actually served the call (the observability span + metrics stamp its provider/model).

#Middleware — wrap every model call

Language-model middleware wraps every model the framework resolves — for generateText, generateObject, streamText, and agents alike — so you add cross-cutting behavior (logging, reasoning extraction, default settings, caching, guardrails) in ONE place without touching call sites. It's the AI SDK's wrapLanguageModel seam, exposed as a process-global stack you install once at boot:

import { setAiMiddleware, loggingMiddleware } from '@voltro/ai'

// Typically in a *.startup.tsx boot hook (or app.config layers):
setAiMiddleware(loggingMiddleware())

Every subsequent call is wrapped; nothing else changes. setAiMiddleware(...) returns the previous stack (restore it in a test), getAiMiddleware() reads it, clearAiMiddleware() empties it. With an empty stack the model is passed through untouched — zero overhead on the default path.

Built-ins (all re-exported from @voltro/ai):

• loggingMiddleware({ log? }) — logs each call's model + token usage (dev default: console; pass log to route into your logger/metrics).
• extractReasoningMiddleware({ tagName }) — split inline <think>…</think> reasoning out of the answer text into the reasoning channel, for models that don't emit native reasoning parts.
• defaultSettingsMiddleware({ settings }) — pin default call settings (temperature, maxOutputTokens, providerOptions) for every call.
• simulateStreamingMiddleware() — make a generate-only model satisfy the streaming path (emits the full text as one delta).

Custom middleware is any object implementing transformParams / wrapGenerate / wrapStream (typed AiMiddleware):

import { setAiMiddleware, type AiMiddleware } from '@voltro/ai'

const redactPII: AiMiddleware = {
  transformParams: async ({ params }) => params,   // scrub params.prompt before it leaves
}
setAiMiddleware(redactPII, loggingMiddleware())    // applied in order — the first entry is outermost

Middleware is server-only — install it where the app boots, never from a browser-safe descriptor.

#Embeddings — a separate axis

Embeddings use their own provider env, not AI_PROVIDER:

AI_EMBED_PROVIDER=openai          # mock (default) | openai | voyage | cohere
AI_EMBED_MODEL=text-embedding-3-small

The default is mock (deterministic, key-free). Real embedding providers (openai / voyage / cohere) resolve their AI-SDK package via a lazy, server-only dynamic import — install the package (e.g. @ai-sdk/openai) to use them. See RAG.

#Provider quirks

• Anthropic 429s burst — burst rate limits trip before the monthly quota. Handle retries with Effect.retry in your executor.
• Mock determinism — every test using useMockAi is isolated; fixtures don't bleed across describe blocks.