API · AI agent

The AI template: a RAG support agent over a docs knowledge base. It exercises the framework's AI surface end-to-end — a vectorEmbedding() docs table that auto-embeds on every write, a defineTool the agent calls to retrieve docs, a real defineAgent / defineAgentExecutor model loop, and a generateObject action for structured output. It boots and discovers every primitive with zero infra and no AI key (store: 'memory'); you only need a provider key to actually RUN the agent or the summarize action. Template id: api-ai.

#Scaffold

voltro create-project acme --api=api-ai

#What ships

apps/acme/api/                          # dir named by the app, not the template
├── app.config.ts                       # type:api, store:'memory', AI env declared
├── package.json                        # adds @voltro/ai to the api deps
├── tsconfig.json
├── README.md
├── database/
│   └── schema.ts                       # actors + tenants (core) + a docs table with vectorEmbedding()
├── agents/
│   ├── support.agent.tsx               # descriptor — browser-safe (name + input)
│   └── support.agent.server.tsx        # executor — system prompt + tools + maxSteps
├── tools/
│   └── searchDocs.tool.tsx             # retrieval tool — nearestNeighbours over docs
├── actions/
│   ├── summarize.action.ts             # descriptor — browser-safe
│   └── summarize.action.server.tsx     # executor — generateObject structured output
└── seeds/
    └── docs.seed.ts                    # boot seed of sample docs (auto-embedded on insert)

No queries or mutations ship — the agent's two routes are SYNTHESIZED (see below). Builds on the same actors + tenants core as api-backend, but the example table is docs (vector-embedded), not notes.

#The agent — descriptor / .server.ts split

Like every rpc primitive, an agent is two files paired by basename: a browser-safe descriptor (*.agent.tsx, defineAgent from @voltro/ai/agent) carrying just name + input, and a server-only executor (*.agent.server.tsx, defineAgentExecutor from @voltro/ai) carrying the system prompt, tools, model, and maxSteps. The descriptor is value-imported into rpcGroup.generated.ts so the web client is typed end-to-end; the executor holds the secrets and the server imports.

#Descriptor — agents/support.agent.tsx

// support.agent.tsx — DESCRIPTOR (browser-safe)
import { defineAgent } from '@voltro/ai/agent'
import { Schema } from 'effect'

export const support = defineAgent({
  name:  'support',
  input: Schema.Struct({
    prompt: Schema.String,
    // Optional locale to steer the system prompt's reply language.
    locale: Schema.optional(Schema.String),
  }),
})

#Executor — agents/support.agent.server.tsx

// support.agent.server.tsx — EXECUTOR (server-only)
import { defineAgentExecutor } from '@voltro/ai'
import { support } from './support.agent'
import { searchDocs } from '../tools/searchDocs.tool'

export default defineAgentExecutor(support, {
  system: (input) =>
    [
      'You are a friendly, precise support agent for this product.',
      'Answer ONLY from the knowledge base. ALWAYS call the `searchDocs`',
      'tool first to retrieve relevant docs, then answer using their',
      'content. If the docs do not cover the question, say so plainly',
      'instead of guessing.',
      `Reply in this locale: ${input.locale ?? 'en'}.`,
    ].join(' '),
  tools:    { searchDocs },
  // No `model` → inherit AI_PROVIDER / AI_MODEL + the key var from env.
  maxSteps: 6,
})

Omitting model makes the agent inherit the provider from env (AI_PROVIDER / AI_MODEL + the provider's standard key var). That keeps the secret server-side and lets the same template run against any provider via .env.

#Two synthesized routes — you write neither

From the agent descriptor the framework SYNTHESIZES two procedures into rpcGroup.generated.ts:

• support.send — an action that appends the user turn and drives a streaming assistant turn (delta-persisted to agent_messages). Wire input: the descriptor's input fields plus threadId + order.
• support.messages — a reactive query (source: 'agent_messages') that streams the persisted turns, including the live streaming:true row being patched, to the client.

The thread tables (agent_threads, agent_messages) are auto-provided and auto-migrated because this app ships an *.agent.tsx — no schema file needed. Drive them from a web client:

// mint a thread id, subscribe to the live feed, then send a turn
type ThreadMessage = {
  readonly id: string
  readonly role: string
  readonly content: string
  readonly streaming: boolean
}

const threadId = `thread_${crypto.randomUUID().replace(/-/g, '')}`
const { data: messages = [] } = useSubscription<ReadonlyArray<ThreadMessage>>(
  'app',
  'support.messages',
  { threadId },
)
const send = useAction('app', 'support.send')
await send.run({ threadId, order: messages.length, prompt: 'How do I reset my password?' })

As support.send writes token deltas to the streaming row, the support.messages subscription re-fires and the client sees each chunk — the live typewriter bubble is just a row whose streaming flag is true. No streaming RPC, no manual ws handling.

#The retrieval tool — tools/searchDocs.tool.tsx

A defineTool (the *.tool.tsx convention) is a named DESCRIPTOR with Schema-typed input / output and NO body, plus a default-exported (input, ctx) => … handler. The framework wires the default export onto the descriptor at discovery, and the agent's tools: { searchDocs } imports the SAME module instance. This file is SERVER-ONLY — it's referenced from the agent executor, never a browser-loaded descriptor, so importing the database handle here is safe.

import { defineTool } from '@voltro/ai'
import type { AppContext } from '@voltro/runtime'
import { Schema } from 'effect'
import { database } from '../database/schema'

const MAX_RESULTS = 5
const SNIPPET_LEN = 280

export const searchDocs = defineTool({
  name:        'searchDocs',
  description:
    'Search the support knowledge base for documents relevant to a question. ' +
    'Returns up to 5 results, each with a title and a short snippet of the body.',
  input:  Schema.Struct({ query: Schema.String }),
  output: Schema.Array(
    Schema.Struct({ title: Schema.String, snippet: Schema.String }),
  ),
})

export default async (
  { query }: { query: string },
  ctx: AppContext,
): Promise<ReadonlyArray<{ title: string; snippet: string }>> => {
  try {
    const rows = await ctx.store.query(
      database.docs.nearestNeighbours(query, MAX_RESULTS).descriptor,
    )
    return rows.map((row) => {
      const body = String((row as { body?: unknown }).body ?? '')
      return {
        title:   String((row as { title?: unknown }).title ?? ''),
        snippet: body.length > SNIPPET_LEN ? `${body.slice(0, SNIPPET_LEN)}…` : body,
      }
    })
  } catch {
    // No AI key / embed failure / cold table → degrade to "no results"
    // rather than aborting the agent's tool loop.
    return []
  }
}

Because docs carries vectorEmbedding(), the STRING overload of nearestNeighbours(query, k) is valid — the runtime embeds query before searching. Any failure (no AI key, embed error) is caught and returns [], so a tool hiccup surfaces to the model as "no results" instead of aborting the whole agent turn.

#The structured-output action — actions/summarize.action.ts + .server.tsx

AI inference is external I/O, so it's an action, not a mutation. This one uses generateObject to take free-text and produce STRUCTURED output — a typed { title, summary, keyPoints, sentiment } object instead of prose. The output Schema is exported from the descriptor so the executor can reuse the SAME schema to constrain the model: one source of truth for both the wire output and the shape the model must satisfy.

// summarize.action.ts — DESCRIPTOR (browser-safe)
import { defineAction } from '@voltro/protocol'
import { Schema } from 'effect'

// The structured shape the model must emit. Top-level `Schema.Struct`
// (generateObject requires a JSON `object` at the root — an array/scalar
// must be wrapped in a field, as `keyPoints` is here).
export const SummaryResult = Schema.Struct({
  title:     Schema.String,
  summary:   Schema.String,
  keyPoints: Schema.Array(Schema.String),
  sentiment: Schema.Literal('positive', 'neutral', 'negative'),
})
export type SummaryResult = Schema.Schema.Type<typeof SummaryResult>

export const summarize = defineAction({
  name:  'support.summarize',
  input: Schema.Struct({
    // The raw text to summarise (a support thread, a doc, a transcript).
    text: Schema.String,
  }),
  output: SummaryResult,
})

// summarize.action.server.tsx — EXECUTOR (server-only)
import { Effect } from 'effect'
import { generateObject } from '@voltro/ai'
import { SummaryResult } from './summarize.action'

const execute = (input: { text: string }) =>
  Effect.gen(function* () {
    const { object } = yield* generateObject({
      schema: SummaryResult,
      system:
        'Summarise the given support text. Produce a short title, a 1-2 sentence ' +
        'summary, the key points as a list, and the overall sentiment.',
      prompt:    input.text,
      maxTokens: 500,
    })
    return object
  })

export default execute

generateObject converts the SummaryResult Effect Schema to JSON Schema for the provider, then decodes the result back so brands/refinements hold. A provider failure or malformed output surfaces as a typed AiError (reason: 'generation' | 'decode') on the Effect failure channel — declare it on the descriptor's error: to surface it typed to the client, or Effect.catchTag('AiError', …) to handle it in the executor.

#Database — auto-embedded docs

The schema declares the two framework core tables plus one vector-embedded knowledge-base table:

// database/schema.ts
import {
  databaseHandle,
  id,
  table,
  text,
  vectorEmbedding,
  type InferRow,
} from '@voltro/database'

// Core tables — required by the audit / tenant mixins.
export const actors = table('actors', {
  id:          id(),
  kind:        text().oneOf(['user', 'serviceAccount', 'apiKey', 'system']),
  displayName: text().nullable(),
})

export const tenants = table('tenants', {
  id:   id(),
  name: text(),
})

// Knowledge base — vectorEmbedding({ from: 'body' }) adds the `embedding`
// vector column + an HNSW index, and registers a re-embed hook that calls
// @voltro/ai's `embed` on every insert/update of `body` — so a plain
// ctx.store.insert('docs', { title, body }) auto-embeds, no app code.
export const docs = table('docs', {
  id:    id({ prefix: 'doc' }),
  title: text(),
  body:  text(),
})
  .with(
    vectorEmbedding({
      from:       'body',
      model:      'text-embedding-3-small',
      dimensions: 1536,
    }),
  )
  // Opt into the matcher engine so subscriptions over `docs` re-fire on write.
  .reactive()

export type Doc = InferRow<typeof docs>

export const database = databaseHandle({ actors, tenants, docs })

dimensions MUST match the embedding model's output width — 1536 is OpenAI's text-embedding-3-small. If you point at a different embedding model, change it to match (e.g. 768 / 3072) or the vector column size won't line up. See Vectors / RAG for nearestNeighbours, hybridSearch, and the vectorEmbedding() mixin in depth.

#The seed — seeds/docs.seed.ts

defineSeed with lifecycle: 'boot' runs once per voltro dev boot, re-running only when the seed file's content fingerprint changes. upsertByUnique keys on title, so re-runs UPDATE rather than duplicate — idempotent. Because docs carries vectorEmbedding({ from: 'body' }), each insert/update AUTO-EMBEDS the body — no embedding code here.

import { defineSeed } from '@voltro/database'

const SAMPLE_DOCS: ReadonlyArray<{ title: string; body: string }> = [
  { title: 'Resetting your password', body: '…' },
  { title: 'Inviting teammates',      body: '…' },
  { title: 'Exporting your data',     body: '…' },
  { title: 'API rate limits',         body: '…' },
]

export default defineSeed({
  id:        'docs',
  name:      'Sample knowledge-base docs',
  lifecycle: 'boot',
  steps: ({ step }) => [
    step('seed-docs', async ({ upsertByUnique }) => {
      let rowsTouched = 0
      for (const doc of SAMPLE_DOCS) {
        await upsertByUnique('docs', { title: doc.title }, { title: doc.title, body: doc.body })
        rowsTouched += 1
      }
      return { rowsTouched }
    }),
  ],
})

The auto-embed needs a provider key. Without one the rows still seed (title + body persist) but the embedding column stays empty. Set a key, then fill the gaps:

voltro embeddings backfill docs --text body --vector embedding
# add --model <m> if you changed the embedding model from the default

This (re-)embeds existing rows the vectorEmbedding() mixin missed — the same command to run after a model change.

#Store + AI provider — env

app.config.ts ships store: 'memory' — fast startup, no Docker, data resets on restart. The vector column is still stored, but ANN falls back to a sequential scan (correct, just unindexed). Switch to store: 'postgres' for a real pgvector HNSW index.

@voltro/ai reads the provider/model/key from env. All are OPTIONAL for booting; they're only needed to RUN the model. The template declares them with defineEnv so they show up in voltro env and .env.example:

// app.config.ts (excerpt)
import { defineEnv, envVar } from '@voltro/env'

export const env = defineEnv({
  LOG_LEVEL:   envVar.enum(['debug', 'info', 'warn', 'error'], { access: 'public', default: 'info' }),
  AI_PROVIDER: envVar.string({ access: 'public', optional: true }),   // e.g. 'openai' | 'anthropic'
  AI_MODEL:    envVar.string({ access: 'public', optional: true }),   // e.g. 'gpt-4o-mini'
  AI_API_KEY:  envVar.string({ access: 'secret', optional: true }),   // server-only
})

export default {
  type:  'api' as const,
  name:  'acmeApi',
  store: 'memory' as const,
  env,
}

# .env — @voltro/ai also accepts the provider-standard var (OPENAI_API_KEY / ANTHROPIC_API_KEY)
AI_PROVIDER=openai
AI_MODEL=gpt-4o-mini
AI_API_KEY=sk-...

Editing .env hard-restarts voltro dev automatically. Run voltro env to see the resolved manifest.

#When to use api-ai vs. the variants

You need…	Pick
RAG agent + tools + structured output (the AI showcase)	api-ai
The smallest generic backend to extend	api-backend
Transactional email wired (React-Email)	api-backend-mail
File storage wired (public + private objects)	api-backend-storage

#Pairs well with

• Any web template — pair support.send / support.messages with a chat UI for the live typewriter feed.
• AI agents — the defineAgent / defineAgentExecutor model loop, synthesized routes, thread persistence.
• AI tools — defineTool shape, how tool bodies reach app services.
• RAG and Vectors — vectorEmbedding(), nearestNeighbours, hybridSearch.

#Anti-patterns

• Hardcoding a model or API key in the descriptor. The model + key live in the .server.tsx executor (or env), never the browser-safe *.agent.tsx. Omitting model to inherit from env keeps the secret server-side and lets one template run against any provider.
• Making the AI call a mutation. AI inference is external I/O — use an action. A mutation runs in a transaction and can't roll back the model call's side effect; summarize is correctly an action.
• Letting a tool failure abort the agent turn. searchDocs catches every error and returns [] so a missing key or embed failure degrades to "no results" instead of crashing the whole tool loop.
• Mismatching dimensions with the embedding model. The vector column width is fixed at declaration. If dimensions doesn't match the model's output width, inserts/queries won't line up — change it together with the model.
• Hand-writing thread/send/list routes. The agent path SYNTHESIZES support.send + support.messages and types them through codegen. Don't reinvent them just to get a typed client.