#Server-owned realtime sessions

defineRealtimeSession mounts a voice/text relay on the existing authenticated WebSocket gateway surface. The browser connects to your app, not the model provider. Both the permanent API key and the SDK-minted session credential stay on your server.

This is a dedicated *.ws.ts gateway route, not the physical RPC subscription socket. It inherits gateway origin checks, subject authentication and credential expiry. Configure the web/reverse proxy to forward this upgrade path to the API, just like any other gateway route.

Custom SDK models must implement both doCreateClientSecret and getWebSocketConfig. These methods are optional in the SDK because other transports exist. The relay closes unsupported sessions with code 4502 before minting credentials or opening a provider socket.

#Declare the server-owned session

// gateways/voice.ws.ts — server-only
import { defineRealtimeSession, resolveRealtimeModel } from '@voltro/ai'
import { Effect } from 'effect'

export default defineRealtimeSession({
  path: '/gateways/voice',
  model: resolveRealtimeModel({ name: 'gateway', model: 'openai/gpt-realtime' }),
  authorize: (subject, model) => Effect.succeed(
    subject.type === 'user'
      && subject.scopes?.includes('ai:voice') === true
      && model.modelId === 'openai/gpt-realtime',
  ),
  sessionConfig: {
    instructions: 'Answer briefly.',
    outputModalities: ['audio', 'text'],
  },
  limits: {
    maxMessageBytes: 256 * 1024,
    maxPendingBytes: 1024 * 1024,
    maxPendingMessages: 32,
    setupTimeoutMs: 15_000,
    maxDurationMs: 10 * 60_000,
  },
})

The SDK reads AI_GATEWAY_API_KEY on the server. Direct OpenAI is also supported by resolveRealtimeModel({ name: 'openai', model: 'gpt-realtime' }), using OPENAI_API_KEY. Model availability remains provider-dependent; the example is not a catalog availability guarantee. The mock and anthropic ProviderConfig branches refuse realtime resolution. You can supply an installed SDK RealtimeModelV4 implementation directly as model instead.

authorize is mandatory and runs with the resolved subject and fixed model identity before minting a credential or opening a provider socket. Authentication alone does not authorize model use or spending. Supply your own tenant/model/ budget policy. The relay does not reserve an AI budget or persist usage receipts. Session configuration is server-owned; the browser cannot select a model, override instructions, set provider options or supply credentials.

Tools are unsupported. Declaring sessionConfig.tools fails at definition time. Client-supplied function results and provider function-call events are refused. This primitive does not execute tools or bypass a tool authorization policy.

#Browser consumption

import { RealtimeServerEvent, type RealtimeClientEvent } from '@voltro/ai/events'
import { Either, Schema } from 'effect'

const decode = Schema.decodeUnknownEither(RealtimeServerEvent)
const url = new URL('/gateways/voice', location.href)
url.protocol = location.protocol === 'https:' ? 'wss:' : 'ws:'
const socket = new WebSocket(url)
let ready = false

socket.onmessage = ({ data }) => {
  const result = decode(JSON.parse(data))
  if (Either.isLeft(result)) return
  const event = result.right
  if (event.type === 'ready') ready = true
  if (event.type === 'text-delta') console.log(event.delta)
  // Feed audio-delta to a player matching the server's outputAudioFormat.
}
socket.onclose = () => { ready = false }

function send(event: RealtimeClientEvent) {
  if (!ready || socket.readyState !== WebSocket.OPEN) return false
  socket.send(JSON.stringify(event))
  return true
}

// Called by a user action, after ready; never replayed automatically.
function ask(text: string) {
  send({ type: 'conversation-item-create', item: { type: 'text-message', role: 'user', text } })
  send({ type: 'response-create' })
}
// On unmount: socket.close(). A later connection starts a NEW session.

Use your normal same-origin authentication cookie. This example never receives a provider URL, key or ephemeral token. It does not implement a microphone encoder, audio player, bearer-token refresh or a reconnection policy.

The browser must wait for ready. It means the server has serialized and written the session configuration before accepting input on the ordered provider connection; it is not a provider acknowledgement. Provider session-created / session-updated timing varies. Early audio is refused, not silently dropped or held indefinitely.

Client events are limited to audio append/commit/clear, user text messages, response create/cancel and conversation-item truncation. Audio is base64 and must match the server-declared input format. Unknown fields, configuration overrides and malformed frames close the connection. The browser-safe RealtimeClientEvent and RealtimeServerEvent schemas/types live at @voltro/ai/events.

Provider events pass through the installed SDK codec and a second schema validation. Only normalized voice/text events are forwarded. Raw payloads, arbitrary conversation-item objects, custom events and provider error details are not exposed. Provider failures use fixed safe close reasons.

#Bounds and lifetime

The values above are the defaults. Byte and message limits cover each relay queue including its in-flight event. The provider socket also enforces a native maximum inbound payload; the app gateway's configurable inbound limit is checked at dispatch time, not a guarantee against native frame allocation. Provider socket output and app-gateway output are bounded. A completed send means a local transport write, not browser receipt.

Setup timeout covers authorization, credential minting, provider connection and initial configuration. Disconnect, credential expiry, provider closure, provider credential expiry or maximum session duration tears down both sides. The relay's Effect scope cancels workers and pending work. The SDK credential mint API has no AbortSignal: cancellation stops waiting and a late result cannot open a socket, but an already-running mint HTTP request may finish.

Close codes used by the relay: 4400 invalid/early input, 4401 session expiry, 4403 policy refusal, 4408 setup timeout, 4409 relay buffer overflow, 4501 provider disconnect and 4502 provider/session failure. Gateway authentication expiry retains its existing 4001 contract; transport limits may close before the relay sees a frame.

There is no automatic retry, reconnect or replay. Audio and response requests may incur costs; a reconnect creates a new session and must be an explicit app decision. A disconnect does not prove the provider cancelled already accepted generation or billing. Use response-cancel for a live response, then close the socket when the component/session ends.