Cost tracking

LLMs are usage-priced, so you need to know how many tokens each call burned. Every @voltro/ai call returns a token usage tally on its result. That's the shipped primitive.

On top of the raw tally, @voltro/ai ships a cost toolkit: a price table + estimateCostUsd, a reactive _voltro_ai_usage ledger (recordAiUsage), spend sums (aiSpendUsd), and a per-tenant budget guard (requireAiBudget → typed AiBudgetExceeded). (@voltro/plugin-audit is separate — it records mutation invocations, not AI calls.)

#Token usage on every call

generateText and generateObject return usage alongside the result:

import { generateText } from '@voltro/ai'
import { Effect } from 'effect'

export default (input: { prompt: string }) =>
  Effect.gen(function* () {
    const { text, usage } = yield* generateText({ prompt: input.prompt })
    // usage = { inputTokens, outputTokens, totalTokens }
    //   each is `number | undefined` (provider-reported)
    return { text, tokens: usage.totalTokens }
  })

const { object, usage } = yield* generateObject({ prompt, schema })
// same usage shape

#Token usage on a streamed run

A streamText run ends with a terminal done event that carries the same usage shape:

import { streamText } from '@voltro/ai'
import { Stream, Effect } from 'effect'

yield* streamText({ prompt }).pipe(
  Stream.runForEach((event) =>
    Effect.sync(() => {
      if (event._tag === 'done') {
        // event.finishReason — 'stop' | 'tool-calls' | 'error' | …
        // event.usage = { inputTokens, outputTokens, totalTokens }
      }
    }),
  ),
)

The done event's usage is the run total across every LLM↔tool round-trip.

#Pricing a call — estimateCostUsd

estimateCostUsd(usage, { model }) turns a token tally into a USD CostBreakdown using the built-in price table (MODEL_PRICING_DEFAULTS, USD per 1M tokens). An unknown model — or the mock provider — prices at zero, so cost accounting never breaks a call.

The defaults cover the major providers — Anthropic (claude-*), OpenAI (gpt-*), and Google Gemini (gemini-*) — so a non-Claude call is priced too (a gpt-4o or gemini-2.5-pro call is a real number, not a silent zero). A gateway creator/model id (openai/gpt-4o) prices by its bare model segment.

import { generateText, estimateCostUsd } from '@voltro/ai'

const model = 'claude-opus-4-8'
const { text, usage } = yield* generateText({ prompt, provider: { name: 'anthropic', model } })
const cost = estimateCostUsd(usage, { model })
// cost = { inputTokens, outputTokens, inputCostUsd, outputCostUsd, totalCostUsd, costSource }
//   costSource: 'estimated' (price-table or zero) | 'gateway' (real reported cost)

Override the price for a model the table doesn't know, or for negotiated / volume pricing:

estimateCostUsd(usage, { model, price: { inputPer1M: 2.5, outputPer1M: 10 } })

#The built-in prices are point-in-time defaults — override them app-wide

MODEL_PRICING_DEFAULTS are public list prices as of January 2026 and WILL drift as providers re-price. Treat them as a sane default for the budget guard + cost dashboard, not a contract. To encode current or negotiated rates once, at boot, without editing the framework, call setModelPricing — a process-global override map merged OVER the defaults (a user entry for a model id wins):

import { setModelPricing } from '@voltro/ai'

// Wire your ai config's `pricing` map through this at boot.
setModelPricing({
  'gpt-4o':          { inputPer1M: 2.5, outputPer1M: 10 },  // corrected list price
  'my-tuned-model':  { inputPer1M: 0.8, outputPer1M: 2.4 }, // a model the defaults don't know
})

Every estimateCostUsd / recordAiUsage / budget call then reads the merged map. priceForModel(model) returns the effective price (or undefined if unknown). For a gateway-routed model, the gateway's REPORTED per-call cost still wins over any static rate (see below).

#Real gateway cost — gatewayCostUsd + actualCostUsd

The static price table only knows the models it lists. A gateway-routed model the table doesn't carry would otherwise estimate to zero — wrong, not just imprecise. The fix: the Vercel AI Gateway reports the ACTUAL per-call cost in the result's provider metadata, and the toolkit prefers it.

gatewayCostUsd(providerMetadata) pulls providerMetadata.gateway.cost (a USD number or numeric string) out of a generate/stream result, returning undefined for a direct provider / the mock (so you fall back to the table):

import { generateText, gatewayCostUsd, recordAiUsage } from '@voltro/ai'

const model = 'openai/gpt-5.5'   // a gateway id the static table doesn't list
const r = yield* generateText({ prompt, provider: { name: 'gateway', model } })
const actualCostUsd = gatewayCostUsd(r.providerMetadata)   // the gateway's real charge, or undefined

const cost = yield* recordAiUsage(ctx.store, {
  tenantId: ctx.request.subject.tenantId,
  provider: 'gateway',
  model,
  operation: 'generateText',
  usage: r.usage,
  actualCostUsd,          // when set → persisted verbatim, costSource: 'gateway'
})
// cost.costSource === 'gateway' (authoritative) when actualCostUsd was present,
// else 'estimated' (price table or zero).

estimateCostUsd(usage, { model, actualCostUsd }) honours the same rule: a present actualCostUsd wins (split across input/output by token share for the breakdown, costSource: 'gateway'); absent, it uses the price table (costSource: 'estimated'). A cost dashboard can flag estimated rows and un-priced (zero) models so you know which numbers are real vs derived.

#The usage ledger — recordAiUsage + aiUsageTable

recordAiUsage(store, {...}) prices a call and writes one row to the _voltro_ai_usage ledger (aiUsageTable), returning the same CostBreakdown. The table is reactive and auto-migrated whenever the app ships any *.agent.tsx; a non-agent app that wants plain-call tracking imports aiUsageTable into its database/index.ts barrel. Cost is stored as integer micro-USD (costMicroUsd, USD × 1e6) — the same "money = integer minor units" rule the billing plugin uses.

import { generateText, recordAiUsage } from '@voltro/ai'

const model = 'claude-opus-4-8'
const { text, usage } = yield* generateText({ prompt, provider: { name: 'anthropic', model } })
const cost = yield* recordAiUsage(ctx.store, {
  tenantId:  ctx.request.subject.tenantId,
  provider:  'anthropic',
  model,
  operation: 'generateText',     // or 'generateObject' | 'streamText' | 'agent' | your own
  usage,
})
// cost.totalCostUsd — surface it without a re-query

A row carries { tenantId, provider, model, operation, agent, inputTokens, outputTokens, costMicroUsd, costSource, calledAt }. costSource is 'gateway' (the actual reported cost) or 'estimated' (price-table / zero) — pass actualCostUsd (see above) to record the real gateway charge.

#Spend + budgets — aiSpendUsd / requireAiBudget

aiSpendUsd(store, { tenantId?, since? }) sums recorded spend (USD). Because aiUsageTable is reactive, a defineQuery with source: '_voltro_ai_usage' that calls it is a live spend meter — the same reactive-query machinery as everything else.

const spentThisMonth = yield* aiSpendUsd(ctx.store, {
  tenantId: ctx.request.subject.tenantId,
  since:    startOfMonth(),
})

requireAiBudget(store, { tenantId, limitUsd, addUsd?, since? }) gates a call against a per-tenant cap — the precedent is billing's requireEntitlement. It fails with a typed, client-marshalable AiBudgetExceeded when reserved + addUsd would exceed limitUsd. Call it BEFORE the provider call (estimate addUsd from the prompt); record the real cost after.

Atomic across replicas — a hard cap, not a soft one. The guard RESERVES addUsd on a single per-tenant counter row (_voltro_ai_budget) via a bounded compare-and-set loop — the same store-level atomic-consume the storage and billing plugins use. So N concurrent calls (same replica or across replicas) reserve exactly the budgeted amount and the rest fail — no overshoot. (This replaces an older check-then-act sum that concurrent callers could all read under-cap and all pass.) The reservation is optimistic: it does NOT auto-release if the provider call later fails, which for a rolling budget is the correct conservative bound. Pass addUsd: 0 for a check-only UI pre-flight that reserves nothing. A rolling since window rotates to a fresh counter (the prior window's row ages out via retention).

import { generateText, requireAiBudget, recordAiUsage, AiBudgetExceeded } from '@voltro/ai'
import { Effect } from 'effect'

export default (input: { prompt: string }, ctx) =>
  Effect.gen(function* () {
    const tenantId = ctx.request.subject.tenantId
    // Refuse if this tenant is already at/over its monthly cap.
    yield* requireAiBudget(ctx.store, { tenantId, limitUsd: 50, addUsd: 0.25, since: startOfMonth() })

    const model = 'claude-opus-4-8'
    const { text, usage } = yield* generateText({ prompt: input.prompt, provider: { name: 'anthropic', model } })
    yield* recordAiUsage(ctx.store, { tenantId, provider: 'anthropic', model, operation: 'generateText', usage })
    return { text }
  })

Declare error: AiBudgetExceeded on the descriptor so the rpc layer surfaces the rejection typed; the client pattern-matches on { _tag: 'AiBudgetExceeded', limitUsd, spentUsd, attemptedUsd } (spentUsd is the amount already reserved on the counter).

#Per-call observability — automatic spans + metrics

Every generateText / generateObject / generateObjectWithTools / streamText call is automatically wrapped in a voltro.ai.call OTel span (attributes ai.provider / ai.model / ai.operation) and records metrics into the framework's global metric registry — the same one @voltro/plugin-prometheus exposes at /metrics and the dashboard reads at /_voltro/inspect/metrics. No wiring needed:

Metric	Type	What
voltro_ai_calls_total	counter	Calls, labelled provider / model / operation / status.
voltro_ai_call_errors_total	counter	Calls that errored.
voltro_ai_call_duration_seconds	histogram	Provider call latency.
voltro_ai_input_tokens_total / voltro_ai_output_tokens_total	counter	Prompt / completion tokens.
voltro_ai_cost_microusd_total	counter	Estimated spend (micro-USD), priced off the merged table.

Labels carry only provider / model / operation ids — never prompt or response content, never a key. Cost here is the estimated figure from the price table (for a live-priced budget cap, use requireAiBudget; for the authoritative gateway charge, use recordAiUsage({ actualCostUsd })).

#Deliberately your call

The toolkit prices + records + gates; a few things stay explicit by design:

• Recording the LEDGER is opt-in per call. The free functions (generateText etc.) have no store or tenant, so they can't self-write the _voltro_ai_usage row — call recordAiUsage where you have ctx (an agent send handler is the natural spot). The metrics above ARE automatic; the durable per-row ledger is the opt-in part.
• The built-in prices are point-in-time list prices. Call setModelPricing to override app-wide, pass price per call for negotiated rates, or actualCostUsd (from gatewayCostUsd) for the gateway's real per-call charge.

For quota tied to billing TIERS (not a raw USD cap), see @voltro/plugin-billing's entitlements — requireEntitlement(ctx, 'aiCalls', n).