), and — if the index declares `queryableFields` — one of those | `SearchFieldRejected` |\n| `engineParams` | only the engine's presentation-only keys (paging, ordering, typo tolerance, highlight shaping) | the key is dropped and logged |\n\n```ts\nimport { SearchFieldRejected, SearchIndexNotFound } from '@voltro/plugin-search'\n```\n\nWhy each one, because the reasoning is what tells you whether your own declaration is tight enough:\n\n- **An unknown index has no spec, therefore no `tenantField`, therefore no tenant clause.** Answering it would run the caller's query unscoped against whatever collection of that name exists on your engine — and a Typesense / Meilisearch / Algolia instance is usually shared with indexes this app never declared. It is refused, not answered empty.\n- **A field name is spliced into the engine's filter DSL.** Typesense's `filter_by` is one flat string that supports `||`, so a crafted name can re-group the boolean tree around the tenant clause appended after it. The identifier pattern is the floor every index gets; `queryableFields` narrows it further to what your UI actually needs. Your `tenantField` does **not** belong in that list — the tenant clause is injected after this check, by the server, and is never a caller's to name.\n- **`engineParams` is not a filter hatch.** It used to be merged last into the engine's params, so a caller could set `filter_by` (Typesense), `filter` (Meilisearch) or `facetFilters` (Algolia) and overwrite the tenant clause. Keys that could select a different document set — those, plus `query_by`, `restrictSearchableAttributes`, `preset`, `pinned_hits`, `enableRules`, … — are dropped. Express a filter as a `filters[]` clause instead: those are validated *and* ANDed with the tenant scope rather than replacing it.\n\nIf your app genuinely needs one more engine key, widen the allowlist **server-side**, where it is a deliberate decision instead of a caller's:\n\n```ts\nsearchPlugin({\n indexes: { /* … */ },\n allowedEngineParams: ['query_by'], // this app trusts callers with this key\n})\n```\n\nDocument-selecting keys stay refused even when listed there — that is the authority the tenant clause holds.\n\n## Backends\n\n| Backend | Notes |\n|---|---|\n| `memoryBackend` (default) | Fully in-process; dev + tests. **Refuses to boot under `NODE_ENV=production`** — see below. |\n| `typesenseBackend` | Optional dep `typesense`. Lazy-loaded. |\n| `meilisearchBackend` | Optional dep `meilisearch`. Lazy-loaded. |\n| `algoliaBackend` | Optional dep `algoliasearch`. Lazy-loaded. |\n\nA backend is the `SearchBackend` interface (`upsert` / `remove` / `query`) — bring your own (OpenSearch, Elastic, …).\n\n### The memory backend refuses to boot in production\n\nThis is a **refusal, not a warning, and not a scale caveat**. Under\n`NODE_ENV=production` on the in-memory backend, `onActivate` throws\n`SearchBackendNotDurable` and the process does not start:\n\n```text\nplugin-search refuses to boot in production on the in-memory backend.\n\nThe memory index lives in THIS process's heap. Two consequences, both silent:\n • every replica holds a different index, so a result depends on which replica served you;\n • the index starts EMPTY after every restart/deploy, and nothing re-seeds it automatically.\n\nConfigure a durable engine in app.config.ts:\n searchPlugin({ backend: { engine: 'typesense', url: …, apiKey: … }, indexes })\n searchPlugin({ backend: { engine: 'meilisearch', url: …, apiKey: … }, indexes })\n searchPlugin({ backend: { engine: 'algolia', appId: …, apiKey: … }, indexes })\nor pass your own `SearchBackend` implementation.\n```\n\n**Running one replica does not make it correct**, which is why the refusal is\nnot conditional on detecting a cluster. Two independent things are wrong with an\nin-process index in a deployment and only one of them is the multi-replica\nstory:\n\n1. **Per-process.** N replicas hold N divergent indexes. Which results you get\n depends on which replica served the request — including *zero hits* for a\n document that demonstrably exists.\n2. **Non-durable.** The index lives in the heap, so every restart and every\n deploy starts EMPTY and nothing re-seeds it: `backfillIndex` is a function\n your app calls, not something the plugin does at boot.\n\n(2) is what a single replica does not fix. It only removes one of the two ways\nthe backend is wrong.\n\nDev is **silent** — the memory backend is exactly right there, and a warning\nthat fires on every `voltro dev` boot is a warning nobody reads.\n\n#### The escape hatch — `singleProcessMemoryIndex`\n\nIf this deployment genuinely is ONE process that re-seeds its index at startup,\nsay so and the boot proceeds:\n\n```ts\nsearchPlugin({\n singleProcessMemoryIndex: true, // exactly one process, and it calls backfillIndex at startup\n indexes: { /* … */ },\n})\n```\n\nTwo things to be clear about before you reach for it:\n\n- **It is a claim about your topology, not a mute switch.** The plugin holds you\n to both halves: exactly one process serves search, and a `*.startup.tsx`\n calls `backfillIndex` for every index — because the index *is* empty after\n each restart until something fills it. The boot logs a `note` at info\n restating what you signed up for.\n- **The plugin checks the claim against reality.** When the instance-membership\n registry reports that a peer replica joined, the plugin logs that the\n declaration is now false — in *any* environment, because a peer announcing\n itself is an observation rather than a guess about `NODE_ENV`:\n\n ```text\n search: replica \"\u003cid>\" joined, but this app declared `singleProcessMemoryIndex: true`.\n That declaration is now false: each replica has its own in-memory index, so search\n results depend on which one serves the request. Configure a durable backend\n (typesense / meilisearch / algolia) or run one process.\n ```\n\n Without the declaration, a peer joining while search is served from a heap\n index warns too — same observation, different wording.\n\n## Dashboard panel\n\nBoth dashboards ship a **Search** panel (api apps): the configured indexes with per-index sync stats (docs synced/removed/dropped, last reindex) + the resolved backend, per-index **drift badges** (`pendingDrift` / `drifted` / last drift time), the **repair queue itself** (the drift-ledger rows, oldest first, with attempt counts and the engine's last error), and two actions — **Reindex** per index (streams the table's current rows back in) and **Resync now** (repairs the drifted rows on demand). Reindex gates on the `canReindexSearch` capability, resync on `canResyncSearch`. Backed by `/_voltro/inspect/plugins/search/{indexes,drift,reindex,resync}`.\n\nThe sync stats behind this panel are **durable and aggregated across replicas** — and counted **in memory first**: each replica buffers its per-event counts and flushes them to the stats table once per window (`sync.statsFlushIntervalMs`, default 5 s; early once `statsFlushMaxBuffered` counts are pending), so an indexed-table write never pays a per-write read+CAS against your primary. A graceful shutdown flushes the tail; a hard crash loses at most the current window of *counters* (never a change — the drift ledger, not these counters, is the durable record of what did not reach the engine). `GET /indexes` drains the buffer before reading, so the panel is always current. `statsFlushIntervalMs: 0` restores one durable write per event. They live in a framework-owned `_voltro_search_stats` table (contributed via `extendSchema.tables`; the plugin declares `store:write`), one row per `(index, replica)`, each bumped with an atomic compare-and-set. `/indexes` aggregates every replica's row — summed under `local` change scope, maxed under `fleet` (see [Under multiple replicas](#under-multiple-replicas)) — and takes the most-recent reindex, so the counts are truthful under multiple instances and survive a restart. Zero-infra dev/tests use an in-process stats store; `bindDataStore` swaps in the durable one at boot, along with the drift ledger.\n\n## Permissions\n\n`store:changes:read` (the ChangeEvent tap) + `store:write` (the durable `_voltro_search_stats` counters and the `_voltro_search_drift` ledger) + `inspect:read` / `inspect:write` (dashboard panel, incl. reindex + resync) + `network:outbound:\u003chost>` (for a remote backend).\n"},"segments":{"0":{"toc":[{"id":"wiring","text":"Wiring","level":2},{"id":"how-sync-works","text":"How sync works","level":2},{"id":"when-the-engine-is-down","text":"When the engine is down","level":2},{"id":"under-multiple-replicas","text":"Under multiple replicas","level":3},{"id":"querying","text":"Querying","level":2},{"id":"query-features","text":"Query features","level":2},{"id":"what-the-server-validates","text":"What the server validates","level":2},{"id":"backends","text":"Backends","level":2},{"id":"the-memory-backend-refuses-to-boot-in-production","text":"The memory backend refuses to boot in production","level":3},{"id":"dashboard-panel","text":"Dashboard panel","level":2},{"id":"permissions","text":"Permissions","level":2}]}},"ran":{"page":true,"segments":[0]}}
VoltroDocumentation

Search

Keep an external search index (Typesense / Meilisearch / Algolia) in sync with your tables via the ChangeEvent tap, query it tenant-scoped through a typed action + hook.

@voltro/plugin-search mirrors your tables into an external search engine. It rides the ChangeEvent tap — every insert/update/delete on a configured table maps into the index automatically; a typed search.query action reads it back, tenant-scoped, with a useSearch hook on the client.

Wiring

// app.config.ts
import { searchPlugin } from '@voltro/plugin-search'

export default {
  type: 'api' as const,
  name: 'api',
  plugins: [
    searchPlugin({
      // backend: defaults to memory — dev + tests only. Under NODE_ENV=production
      // the memory backend REFUSES TO BOOT (see "The memory backend refuses to
      // boot in production" below). Name a durable engine for a deployment:
      // backend: { engine: 'typesense', url, apiKey } | { engine: 'meilisearch', … } | { engine: 'algolia', appId, apiKey }
      indexes: {
        posts: {
          index: 'posts',
          tenantField: 'tenantId',                       // index documents carry the tenant → query scopes by it
          queryableFields: ['title', 'body', 'status'],  // optional: fields a caller may filter/facet/highlight on
          map: (row) => ({ id: String(row.id), title: String(row.title), body: String(row.body) }),
        },
      },
      // allowedEngineParams: ['query_by'],               // optional: widen the engineParams allowlist (see below)
    }),
  ],
}

How sync works

The plugin declares onChangeEvent. On every committed write to a configured table, applyChange maps the row via map(row) and upserts (insert/update) or removes (delete) the index document. No *.subscribe.ts, no manual indexing calls — the tap is the single sync path, and it runs under both voltro dev and voltro serve.

For pre-existing rows, backfillIndex(backend, spec, rows) indexes the rows you supply (call it from a *.startup.tsx or a one-off script) — or use the Reindex button / POST /reindex inspect endpoint, which reads the table's current rows for you. The endpoint streams the table (keyset-paginated) and upserts one bounded page at a time — sync.reindexBatchSize, default 1000 rows — so memory stays flat no matter how large the table is. backfillIndex itself stays a plain array API for small explicit seeds.

When the engine is down

The database commit has already happened by the time the tap runs, so a failed index write is drift: the row is in your database and missing from — or stale in — your index. That is not left to a log line.

  1. Retry. A failure the backend marks transient (network, engine unavailable, 5xx, rate limited) is retried inside the tap's own Effect with capped exponential backoff. A permanent failure — an unsupported query shape, a map(row) that throws on one row's shape — is not retried: repeating it cannot succeed, and it lands in step 2 immediately.
  2. Record. A change that outlives the retry is written to the framework-owned _voltro_search_drift table, one row per (index, source row). Nothing is lost at that point: the entry lives in your own database — the thing that just committed successfully — while the search engine is what is down.
  3. Repair. A cluster-coordinated sweep re-reads each recorded row from the database and re-derives its document. It never replays the failed event, and that is what makes repair order-free and idempotent: a row updated three times during an outage converges in one pass, and a row deleted since the failure converges to a removal.

The one case that is still a loss is the engine failing and the ledger write failing — and that one fails the tap loudly rather than reporting success.

Every number here is yours to set (defaults shown):

searchPlugin({
  indexes: { /* … */ },
  sync: {
    retries: 5,               // retries after the first attempt; transient failures only. 0 disables retry
    retryBaseDelayMs: 200,    // first backoff — doubles per attempt
    retryMaxDelayMs: 10_000,  // ceiling for any single wait
    resyncIntervalMs: 60_000, // repair-sweep interval. 0 turns the sweep off (POST /resync still repairs on demand)
    resyncBatchSize: 200,     // max ledger entries repaired per sweep
    reindexBatchSize: 1000,   // rows per page for POST /reindex (streamed keyset walk — memory stays flat)
    statsFlushIntervalMs: 5_000, // buffered sync-counter flush cadence. 0 = one durable write per event
    statsFlushMaxBuffered: 1000, // flush early once this many counts are pending
  },
})

Drift is visible without reading logs:

Endpoint Shows
GET /_voltro/inspect/plugins/search/indexes per index: dropped, pendingDrift, lastDriftAt, drifted — plus the retry policy actually in force
GET …/search/drift the failing rows themselves, oldest first, with their last error and attempt count
POST …/search/resync runs a repair pass now ({ scanned, repaired, failed })

An entry whose attempts keeps climbing is telling you something a retry cannot fix — a map(row) that throws on that row, a document the engine rejects. That is the case to look at by hand; everything else drains on its own.

_voltro_search_drift is applied by the declarative differ on voltro db apply and on a voltro dev boot, on every dialect — there is nothing to migrate.

Under multiple replicas

Index writes and index counters are deliberately treated differently when your store's change scope is fleet (every replica receives every change — postgres LISTEN/NOTIFY CDC, for instance):

  • The write runs on every replica. An upsert/remove of the same document is idempotent, so a duplicate costs write amplification — while electing a single writer would cost a lost update whenever that replica dies mid-change, and would import the leadership-gap window with it.
  • The count runs everywhere too, but is aggregated as a maximum. Each replica's stats row is already a fleet-wide count of the same changes, so /indexes takes the highest rather than the sum — the panel reports one sync per change, not one per replica. Under local scope only the replica that actually made the write counts (peers see an echo), and the rows are summed. Neither path needs a leader, so neither has a window in which counting stops.

Querying

search.query is a typed action (not a streaming query — search results aren't reactive):

import { useSearch } from '@voltro/plugin-search/web'

const { results, facets, run, pending } = useSearch('posts')
run('voltro effect')                       // → results scoped to the caller's tenant
run('voltro effect', { limit: 20, offset: 20 })   // offset paging — page 2

results are the hit documents; facets are the facet counts from the last run. The query is automatically filtered to the caller's tenantId (from the resolved Subject) against the index's tenantField — no cross-tenant leak.

run(q, { limit, offset }) pages the results: offset skips leading hits, limit bounds the window. It maps to each engine's native paging (memory slice, Typesense/Meilisearch offset, Algolia offset+length).

Query features

Every option is plain JSON and round-trips through the action's Schema:

import { memoryBackend } from '@voltro/plugin-search'

const backend = memoryBackend()
const res = await backend.query('posts', {
  q: 'kubernetis',                                   // matches the typo below
  limit: 20,
  offset: 20,                                        // offset paging
  fuzziness: 'auto',                                 // typo tolerance (or a number: max edits, 0 = exact)
  filters: [
    { field: 'status', op: 'neq', value: 'draft' },  // negation
    { field: 'score', op: 'gte', value: 50 },        // numeric range
    { field: 'publishedAt', op: 'lt', value: '2026-06-01T00:00:00Z' }, // date range
    { field: 'tag', op: 'in', value: ['a', 'b'] },   // set membership
  ],
  facets: ['status', 'tag'],                         // facet counts per field
  highlight: { fields: ['title'], preTag: '<mark>', postTag: '</mark>' },
  engineParams: { num_typos: 1 },                    // escape hatch → forwarded verbatim to the engine
})
// res.hits: [{ doc, highlights? }, …]   res.facets: { status: { open: 12, done: 3 }, … }
  • filters — a list of clauses ANDed together, each { field, op, value } with op ∈ eq | neq | gt | gte | lt | lte | in | nin. Range (gt/lte/…) and negation (neq/nin), not equality-only.
  • facets — per-value counts for the named fields (over the full matched set, before paging).
  • highlight — matched-term snippets per field, returned as hit.highlights[field].
  • fuzziness — a max edit distance (0 = exact) or 'auto'.
  • engineParams — the engine's presentation-only params (see the next section). Ignored by the memory backend.

Native support degrades honestly: memory, Typesense, Meilisearch and Algolia all do filters/facets/highlighting; Typesense honors a numeric typo count (num_typos 0–2) while Meilisearch and Algolia only toggle typo tolerance on/off (fuzziness: 0 disables it, other values keep their built-in tolerance). A backend op fails with a typed SearchBackendError.

What the server validates

search.query is a public wire surface, and three of its inputs — the index name, the field names, and engineParams — become the search engine's control plane. The plugin validates all three server-side, before the engine sees them, and refuses with a typed error rather than answering a query it cannot scope:

Input Rule On violation
index must be one of the indexes you declared in searchPlugin({ indexes }) SearchIndexNotFound
filters[].field, facets[], highlight.fields[] a plain field path (^[A-Za-z_][A-Za-z0-9_.]*$), and — if the index declares queryableFields — one of those SearchFieldRejected
engineParams only the engine's presentation-only keys (paging, ordering, typo tolerance, highlight shaping) the key is dropped and logged 
import { SearchFieldRejected, SearchIndexNotFound } from '@voltro/plugin-search'

Why each one, because the reasoning is what tells you whether your own declaration is tight enough:

  • An unknown index has no spec, therefore no tenantField, therefore no tenant clause. Answering it would run the caller's query unscoped against whatever collection of that name exists on your engine — and a Typesense / Meilisearch / Algolia instance is usually shared with indexes this app never declared. It is refused, not answered empty.
  • A field name is spliced into the engine's filter DSL. Typesense's filter_by is one flat string that supports ||, so a crafted name can re-group the boolean tree around the tenant clause appended after it. The identifier pattern is the floor every index gets; queryableFields narrows it further to what your UI actually needs. Your tenantField does not belong in that list — the tenant clause is injected after this check, by the server, and is never a caller's to name.
  • engineParams is not a filter hatch. It used to be merged last into the engine's params, so a caller could set filter_by (Typesense), filter (Meilisearch) or facetFilters (Algolia) and overwrite the tenant clause. Keys that could select a different document set — those, plus query_by, restrictSearchableAttributes, preset, pinned_hits, enableRules, … — are dropped. Express a filter as a filters[] clause instead: those are validated and ANDed with the tenant scope rather than replacing it.

If your app genuinely needs one more engine key, widen the allowlist server-side, where it is a deliberate decision instead of a caller's:

searchPlugin({
  indexes: { /* … */ },
  allowedEngineParams: ['query_by'],   // this app trusts callers with this key
})

Document-selecting keys stay refused even when listed there — that is the authority the tenant clause holds.

Backends

Backend Notes
memoryBackend (default) Fully in-process; dev + tests. Refuses to boot under NODE_ENV=production — see below.
typesenseBackend Optional dep typesense. Lazy-loaded.
meilisearchBackend Optional dep meilisearch. Lazy-loaded.
algoliaBackend Optional dep algoliasearch. Lazy-loaded.

A backend is the SearchBackend interface (upsert / remove / query) — bring your own (OpenSearch, Elastic, …).

The memory backend refuses to boot in production

This is a refusal, not a warning, and not a scale caveat. Under NODE_ENV=production on the in-memory backend, onActivate throws SearchBackendNotDurable and the process does not start:

plugin-search refuses to boot in production on the in-memory backend.

The memory index lives in THIS process's heap. Two consequences, both silent:
  • every replica holds a different index, so a result depends on which replica served you;
  • the index starts EMPTY after every restart/deploy, and nothing re-seeds it automatically.

Configure a durable engine in app.config.ts:
  searchPlugin({ backend: { engine: 'typesense',    url: …, apiKey: … }, indexes })
  searchPlugin({ backend: { engine: 'meilisearch',  url: …, apiKey: … }, indexes })
  searchPlugin({ backend: { engine: 'algolia',      appId: …, apiKey: … }, indexes })
or pass your own `SearchBackend` implementation.

Running one replica does not make it correct, which is why the refusal is not conditional on detecting a cluster. Two independent things are wrong with an in-process index in a deployment and only one of them is the multi-replica story:

  1. Per-process. N replicas hold N divergent indexes. Which results you get depends on which replica served the request — including zero hits for a document that demonstrably exists.
  2. Non-durable. The index lives in the heap, so every restart and every deploy starts EMPTY and nothing re-seeds it: backfillIndex is a function your app calls, not something the plugin does at boot.

(2) is what a single replica does not fix. It only removes one of the two ways the backend is wrong.

Dev is silent — the memory backend is exactly right there, and a warning that fires on every voltro dev boot is a warning nobody reads.

The escape hatch — singleProcessMemoryIndex

If this deployment genuinely is ONE process that re-seeds its index at startup, say so and the boot proceeds:

searchPlugin({
  singleProcessMemoryIndex: true,   // exactly one process, and it calls backfillIndex at startup
  indexes: { /* … */ },
})

Two things to be clear about before you reach for it:

  • It is a claim about your topology, not a mute switch. The plugin holds you to both halves: exactly one process serves search, and a *.startup.tsx calls backfillIndex for every index — because the index is empty after each restart until something fills it. The boot logs a note at info restating what you signed up for.

  • The plugin checks the claim against reality. When the instance-membership registry reports that a peer replica joined, the plugin logs that the declaration is now false — in any environment, because a peer announcing itself is an observation rather than a guess about NODE_ENV:

    search: replica "<id>" joined, but this app declared `singleProcessMemoryIndex: true`.
That declaration is now false: each replica has its own in-memory index, so search
results depend on which one serves the request. Configure a durable backend
(typesense / meilisearch / algolia) or run one process.

    Without the declaration, a peer joining while search is served from a heap index warns too — same observation, different wording.

Dashboard panel

Both dashboards ship a Search panel (api apps): the configured indexes with per-index sync stats (docs synced/removed/dropped, last reindex) + the resolved backend, per-index drift badges (pendingDrift / drifted / last drift time), the repair queue itself (the drift-ledger rows, oldest first, with attempt counts and the engine's last error), and two actions — Reindex per index (streams the table's current rows back in) and Resync now (repairs the drifted rows on demand). Reindex gates on the canReindexSearch capability, resync on canResyncSearch. Backed by /_voltro/inspect/plugins/search/{indexes,drift,reindex,resync}.

The sync stats behind this panel are durable and aggregated across replicas — and counted in memory first: each replica buffers its per-event counts and flushes them to the stats table once per window (sync.statsFlushIntervalMs, default 5 s; early once statsFlushMaxBuffered counts are pending), so an indexed-table write never pays a per-write read+CAS against your primary. A graceful shutdown flushes the tail; a hard crash loses at most the current window of counters (never a change — the drift ledger, not these counters, is the durable record of what did not reach the engine). GET /indexes drains the buffer before reading, so the panel is always current. statsFlushIntervalMs: 0 restores one durable write per event. They live in a framework-owned _voltro_search_stats table (contributed via extendSchema.tables; the plugin declares store:write), one row per (index, replica), each bumped with an atomic compare-and-set. /indexes aggregates every replica's row — summed under local change scope, maxed under fleet (see Under multiple replicas) — and takes the most-recent reindex, so the counts are truthful under multiple instances and survive a restart. Zero-infra dev/tests use an in-process stats store; bindDataStore swaps in the durable one at boot, along with the drift ledger.

Permissions

store:changes:read (the ChangeEvent tap) + store:write (the durable _voltro_search_stats counters and the _voltro_search_drift ledger) + inspect:read / inspect:write (dashboard panel, incl. reindex + resync) + network:outbound:<host> (for a remote backend).