VoltroDocumentation

Overview

How Voltro treats AI — agents, tools, streaming, RAG — all primitives over the same WebSocket as the rest of the framework.

AI in Voltro isn't a library you bolt on. It's a primitive: agents are files, tools are files, embeddings are columns, streaming is the same WebSocket that carries queries + mutations.

The implementation is the Vercel AI SDK wrapped behind the @voltro/ai surface so the provider choice (Anthropic, OpenAI, the Vercel AI Gateway, mock-for-tests) is one env var — or a per-agent / per-call override carrying its own key.

The model

               ┌───────────────────────────────────────┐
               │  Vercel AI SDK                        │
               │   anthropic / openai / gateway / mock │
               └───────────────────┬───────────────────┘
                                   │
                                   ▼
┌──────────────────────────────────────────────────────────────────────┐
│  @voltro/ai                                                          │
│    generateText(...) / generateObject(...)                           │
│    streamText(...)                                                   │
│    embed(...) / embedMany(...)                                       │
└────────────────┬────────────────────────────────────┬────────────────┘
                 │                                    │
                 ▼                                    ▼
┌────────────────────────────────┐   ┌─────────────────────────────────┐
│  *.agent.tsx (descriptor)      │   │  *.tool.tsx                     │
│   defineAgent({ name, input }) │   │   defineTool({ name, input,     │
│  *.agent.server.tsx (executor) │   │     output })                   │
│   defineAgentExecutor(desc, {  │   │   typed + wired on the executor │
│     system, tools, model })    │   │                                 │
└────────────────────────────────┘   └─────────────────────────────────┘

What's in this section

  • Providers — Anthropic, OpenAI, the Vercel AI Gateway, mock; per-config keys (BYOK), switching at runtime, model-wrapping middleware
  • Agents — *.agent.tsx shape, system prompts, tool wiring
  • Tools — *.tool.tsx shape, validation, side effects
  • Streaming — tokens-over-WebSocket, client hooks, backpressure
  • RAG — pgvector, embedding mixin, hybrid search
  • Cost tracking — the token usage returned on every call

Why one surface

The Vercel AI SDK is excellent. It also rev'd its public API three times in 2024. Wrapping it gives us:

  1. One break when the SDK changes shape — we update @voltro/ai, your code keeps working.
  2. One mock implementation for tests — AI_PROVIDER=mock makes every call deterministic.
  3. Provider portability — switch the provider with one env var, no call-site changes.

Conceptual differences vs. raw SDK

Vercel AI SDK @voltro/ai
generateText(...) generateText({ prompt, system }) returns an Effect
streamText(...) streamText({ prompt, system }) returns an Effect Stream
Embeddings via embedMany([texts]) embed(text) / embedMany(texts)
Tool definitions are loose objects defineTool({ name, input, output, … }) with Schema
Provider is hardcoded in code AI_PROVIDER env var

When NOT to use Voltro's AI surface

  • You need bleeding-edge SDK features before they land in @voltro/ai. Drop down to the raw SDK via import { anthropic } from '@ai-sdk/anthropic'.
  • You're calling AI from outside an executor. Workers, CLI scripts, etc. can still use @voltro/ai directly when they provide the right env/config.

For 95% of app code inside actions, streams, workflows, and agent helpers, use @voltro/ai instead of importing provider SDKs directly. The portability and test mock surface stay in one place.