Troubleshooting

Every refuse-to-boot from the planner includes a structured fix hint. This page is the comprehensive catalog of those hints + the schema edit each one wants.

#"NOT NULL column on a table whose row count is unknown"

Full error:

auto-migrate: REFUSED — 1 blocked operation(s):
  - add-column [users]: NOT NULL column on a table whose row count is unknown
    fix: declare `email: text().backfill(sql`...`)` OR `.default(value)` so existing rows survive the migration

What happened: you added a non-nullable column to a populated table without telling the planner how to populate it for existing rows.

Fix — pick ONE:

// Option A — constant default (planner emits ADD COLUMN ... NOT NULL DEFAULT value)
plan: text().default('free'),

// Option B — SQL expression (planner emits ADD nullable → UPDATE → SET NOT NULL)
email: text().backfill(sql`'unknown-' || id || '@local'`),

// Option C — JS function (slower, only when SQL can't express what you need)
embedding: text().backfill(async (row) => embed(row.title)),

// Option D — make it nullable
bio: text().nullable(),

Decision rubric in backfill.md.

#"column missing from declared schema" (DROP COLUMN refused)

Full error:

✗ ALTER TABLE users DROP COLUMN legacyField
  ! fix: if intentional, add `legacyField: dropped()` to the schema. If a typo, restore the field

What happened: the live DB has a column the declared schema doesn't reference. Could be deliberate (you want to drop it) or accidental (someone deleted the field from the schema by mistake).

Fix — pick ONE:

// Option A — declare intent. Column goes away on next apply.
export const users = table('users', {
  id:           id(),
  legacyField:  dropped(),   // ← explicit. Planner allows the drop.
})

// Option B — typo, restore the field.
export const users = table('users', {
  id:           id(),
  legacyField:  text(),
})

// Option C — use VOLTRO_DESTRUCTIVE_OK for one-off applies (loud warning):
// VOLTRO_DESTRUCTIVE_OK=1 voltro db apply

#"table missing from declared schema" (DROP TABLE refused)

✗ DROP TABLE oldUsersTable
  ! fix: if intentional, set VOLTRO_DESTRUCTIVE_OK=1 OR add a file-based migration. If a typo, restore the table declaration

Fix:

// Option A — restore the table declaration (probably the right answer if surprised)
export const oldUsersTable = table('oldUsersTable', { id: id(), ... })

// Option B — file-based migration that moves data out + then drops.
// The file context is { sql, log, appliedAt } — no store; use sql:
// migrations/20260415_retire_old_users.ts
export default migration({
  id:   '20260415_retire_old_users',
  description: 'Move oldUsersTable rows into users, then drop it.',
  up:   async ({ sql }) => {
    await sql.unsafe(`INSERT INTO users (id, ...) SELECT id, ... FROM "oldUsersTable"`)
    await sql.unsafe(`DROP TABLE "oldUsersTable"`)
  },
  down: async ({ sql }) => { /* recreate + restore as far as possible */ },
})

// Option C — one-off destructive apply:
// VOLTRO_DESTRUCTIVE_OK=1 voltro db apply --note 'retiring oldUsersTable per ticket #...'

Two tables this never proposes dropping:

• actors — the framework-provided audit subject. You don't declare an actors.entity.ts; db plan / db apply auto-include the built-in actors in the declared set, so it's never a DROP candidate. (Declare your own actors with extra columns and that takes precedence.)

• Your own UNMANAGED infra tables — a table you keep outside the Voltro schema (a migration id-map, a legacy audit table). List them in VOLTRO_DB_IGNORE_TABLES (comma-separated) and the diff leaves them alone instead of planning a DROP:

  VOLTRO_DB_IGNORE_TABLES=_strapi_id_map,_legacy_audit voltro db apply

  The same env var is honoured by the voltro dev boot auto-migrate, not just the db plan / db apply CLI — set it in the app's environment and the boot diff leaves the listed tables alone too, so a fresh clone with a Strapi→Voltro _strapi_id_map sitting in the DB won't refuse-to-boot on a drop-table. (The framework already self-excludes its own _voltro_* / cluster_* runtime tables; this is the user list on top of that.)

#A migrate / apply DDL statement failed — find which one

When voltro migrate / db apply hits a DDL error, the CLI names the failing statement plus the driver's fields, not just a stack:

═══ migrate: statement failed ═══
statement: CREATE INDEX "users_orgId_idx" ON "users" ("orgId")
db.message: column "orgId" does not exist
db.code: 42703

A column … does not exist on a CREATE INDEX usually means the column was never added to an EXISTING table: voltro migrate (auto-migrate) is CREATE TABLE IF NOT EXISTS — it does NOT ADD COLUMN to a table that already exists. To evolve an existing table's columns, use the declarative path (voltro db plan → db apply), which orders ADD COLUMN before the index. Set VOLTRO_MIGRATE_DEBUG=1 to trace every statement as it executes.

#"users.givenName looks like a new required column on a populated table"

Full error:

✗ ALTER TABLE users ADD COLUMN givenName text  # NOT NULL column on a table whose row count is unknown

What probably happened: you renamed firstName → givenName in the schema, but didn't tell the planner it's a rename. The planner sees firstName gone + givenName new + classifies each separately.

Fix:

export const users = table('users', {
  id:        id(),
  givenName: text().renamedFrom('firstName'),
})

The planner folds the diff into one ALTER TABLE users RENAME COLUMN firstName TO givenName, classified safe. After the rename is applied in every env, the marker can be removed (covered in rename-and-drop.md).

#"Schema fingerprint mismatch" (prod refuse)

Full error:

[voltro:start] auto-migrate: SCHEMA FINGERPRINT MISMATCH —
  declared = a8f2c9d10b3f4e62
  live     = 8f507ba1e1aadad5
Run `voltro db apply --plan plan.json` from the deploy pipeline before serving.
exit 1

What happened: the production runtime checked its declared schema's fingerprint against the latest _voltro_migration_plans.fingerprint and they don't match. The framework refuses to start serving because it doesn't know what to do — auto-apply on prod isn't allowed (see prod-pipeline.md).

Fix:

1. Preview the plan against the prod DB shape (NOT prod credentials — use a staging-replica snapshot):

DB_URL=<staging-snapshot-url> voltro db plan

2. Review the printed plan in the PR.

3. Run apply from the CI/CD step (a one-shot job with migration credentials and NODE_ENV unset / staging — voltro db apply re-diffs live and refuses on NODE_ENV=production):

voltro db apply --note 'PR #1234'

4. Re-deploy. The new boot's fingerprint check passes.

Note: the boot-mismatch message itself prints voltro db apply --plan plan.json, but --plan is not a real flag — the prod apply is a plain voltro db apply that recomputes the diff.

If the cause is drift (someone DDL'd prod manually), see drift.md for reconciliation paths.

#"duplicate column name in plan" (logically invalid plan)

db apply: refusing — plan contains 2 ops targeting users.email (CREATE + DROP)
This usually means the planner couldn't determine the correct order.
Hand-edit the plan JSON or use a file-based migration to express the intent explicitly.

What happened: the diff produced both an ADD and DROP for the same column → ambiguous intent.

Fix: it's almost always a schema edit ordering problem. Either:

• The schema was edited twice + both edits are in the diff (rebase the PR; squash the two commits)
• A column was renamed + another column was added with the same name (use .renamedFrom() on the second)
• A file-based migration is racing the planner (sequence them differently — file before planner)

#"VOLTRO_DESTRUCTIVE_OK relaxes lossy ops only; plan also contains rename-without-marker"

auto-migrate: REFUSED — 2 blocked operation(s):
  - drop-column [users.legacy]: lossy
    fix: if intentional, add `legacy: dropped()` to the schema
  - rename-column [users.givenName from firstName]: rename without marker
    fix: declare `.renamedFrom('firstName')` on the new column

VOLTRO_DESTRUCTIVE_OK=1 was set but at least one blocked op is NOT lossy.
The flag only relaxes lossy ops; other refuse cases (rename, NOT-NULL-no-backfill, multi-step) stay firm.

What happened: you reached for VOLTRO_DESTRUCTIVE_OK=1 to bypass a refuse, but the plan has a non-lossy refuse too. The flag is intentionally narrow.

Fix: address the non-lossy refuse first (add the rename annotation in the example above). Then the flag relaxes the remaining lossy op.

#"live introspection failed; cannot diff"

db plan: live introspection failed
  cause: Connection refused at localhost:5432

What happened: the framework can't reach the DB. The planner needs a live introspection to compute the diff.

Fix:

• Check the DB is running (docker ps, systemctl status postgres)
• Check the connection URL: echo $DB_URL matches what the DB expects
• Check credentials: psql $DB_URL -c 'SELECT 1' should succeed
• If using cloud, check the inspectToken: curl -H "Authorization: Bearer $TOKEN" "$APP_URL/_voltro/inspect/app" should return JSON

#"advisory lock held; refusing to wait"

db apply: refusing — advisory lock 8732891 is held by another process (pid 4892)
This usually means another `voltro db apply` is running. Wait for it to finish or kill the holder.

Fix:

• If a real apply is running elsewhere, wait
• If the holder is stuck (pid 4892 died without releasing):
  • Postgres: SELECT pg_advisory_unlock(8732891); (run as the same user that acquired)
  • Or kill the postgres backend: SELECT pg_terminate_backend(<pid>)
  • For MySQL: SELECT RELEASE_LOCK('voltro_migration') from the same connection (different connection won't release)

The lock is per-database-cluster, not per-deploy. Two prod regions hitting the same DB cluster race; the second blocks until the first releases.

#voltro dev boot hangs at "auto-migrate: planning schema" (0/1, no error)

The boot-time auto-migrate takes the same migration advisory lock as db apply. If a prior boot crashed while holding it (its DB connection still open) or a sibling pod holds it, the boot would otherwise wait on the lock — the pod sits at auto-migrate: planning schema, readiness never flips, and no error line prints.

The boot now fails fast instead of hanging: it polls the lock to a deadline (default 30s) and then aborts with a clear message rather than blocking forever.

could not acquire the postgres migration advisory lock within 30s.
Another migration is in progress, or a prior boot crashed while holding it.

Fix:

• A crashed process's session-level advisory lock is released the moment its DB connection closes — so a truly dead holder frees the lock on its own; just restart.
• If a live-but-stuck backend holds it, find + terminate it: SELECT pid, query FROM pg_stat_activity WHERE query LIKE '%advisory%' → SELECT pg_terminate_backend(<pid>).
• Long, legitimate migrations on a big schema can outlast 30s — raise the ceiling with VOLTRO_MIGRATION_LOCK_TIMEOUT_MS (milliseconds).

Related: if the boot instead REFUSES with a drop-table blocker for a table you want to keep (a _strapi_id_map-style leftover), that's the "table missing from declared schema" case — VOLTRO_DB_IGNORE_TABLES unfreezes it.

#Variant: it hangs even with the lock free (large / FK-dense schema)

Same symptom, different cause. If nothing else holds the lock and the boot still sits at auto-migrate: planning schema, the schema introspection is the bottleneck — the step that reads the live database shape before diffing. It only runs on a real diff (a no-diff boot skips it via the schema fingerprint), which is why adding a single column can trigger it while an unchanged restart boots fine.

The cause is almost always a large, foreign-key-dense schema (hundreds of tables, thousands of FKs). Introspection reads foreign keys and primary keys directly from pg_catalog (index-backed, filter pushed down) rather than the information_schema constraint views — those can't push the per-batch table filter down, so each batch re-scans the whole catalog. On a 500-table / 2600-FK schema that is the difference between > 2 minutes (hangs) and well under a second.

If introspection ever degenerates again it fails fast instead of hanging: every introspection statement runs under a statement_timeout (default 30s), so a runaway query aborts with an actionable error rather than freezing the pod at 0/1.

schema introspection exceeded VOLTRO_INTROSPECT_TIMEOUT_MS (30000ms) — the schema is
very large / FK-dense or the database is slow.

Fix:

• Raise the ceiling for a legitimately huge schema with VOLTRO_INTROSPECT_TIMEOUT_MS (milliseconds; 0 disables it entirely).
• Prefer a direct (non-pooler) connection for migrations via DB_DIRECT_URL — so a large introspection response isn't mis-framed by a transaction-mode pooler.
• VOLTRO_DB_IGNORE_TABLES does not help here — it filters the diff, which runs after introspection; the introspection cost is independent of it.

#A MySQL/MariaDB apply failed midway

voltro db apply on postgres / mssql is ATOMIC: every op runs in ONE transaction, so a failure on op N rolls the WHOLE plan back — nothing is committed, no half-applied schema. (online-required CREATE INDEX CONCURRENTLY ops run after the commit — they can't be in a transaction — so a failure THERE can leave the index half-built; re-apply finishes it.)

Run migrations through a SESSION connection, not a transaction-mode pooler. Because the whole plan is one transaction, a large apply (many ops + big backfill UPDATEs + index builds) is ONE long-lived transaction. A transaction-mode pooler (Supabase Supavisor on :6543, PgBouncer in transaction mode) can't hold a multi-statement transaction reliably and will abort it — surfacing as an opaque Failed to execute statement (at sql.transaction). Point DB_URL at the direct / session connection (:5432, or a session-mode pooler) for db apply; raise statement_timeout for that session if a single index build is slow. This is the same constraint every migration tool has (Prisma/Drizzle/etc.) — the transaction pooler is for app traffic, the direct connection is for migrations. The failing statement itself is now logged with its SQL + db.code (e.g. 57014 statement timeout) so you can see which op stalled.

MySQL and MariaDB (and sqlite / turso) implicit-commit every DDL statement, so THERE a plan that fails on op N leaves ops 1..N-1 committed. There is no --resume / --abort flag and no per-op partial-status tracking — the apply records a row only on full success.

Recovery is just to re-run the apply: voltro db apply re-diffs the declared schema against the current (half-applied) live shape and emits only the ops that are still missing. Fix the cause of the failed op first (e.g. the ER_DUP_KEYNAME that stopped op N), then:

voltro db apply --note 'completing partial apply after fixing op N'

If a deploy reverted the code that referenced the half-applied schema, the re-diff naturally reflects the new declared shape — no separate abort step is needed; the next voltro db plan already shows the correct remaining work. See multi-dialect.

#relation "..._uq" already exists (42P07) on re-apply

A composite (multi-column) .unique([a, b]) constraint that ALREADY exists in the DB is now introspected (postgres reads it back from pg_constraint), so db apply matches it against the declared schema and emits nothing. On older builds it wasn't read back, so the planner re-emitted ADD CONSTRAINT … UNIQUE for the existing one → 42P07 relation "<name>_uq" already exists, and the plan never reached "up to date". If you see this, update the framework. (Single-column .unique() was never affected — it round-trips via the column's unique flag.)

#db plan keeps showing CREATE INDEX for indexes that already exist

If db plan always lists add-index for expressionIndex(...) / jsonIndex(...) indexes that demonstrably exist in the DB — and you never see a matching drop-index — that's an introspection gap (now fixed). On postgres an expression key carries a 0 in pg_index.indkey (it has no backing column), and the old introspect query INNER-joined pg_attribute on the column → the whole index disappeared from the live snapshot. The declared index then had nothing to match → re-emitted every run, but never converged to "up to date". (CREATE INDEX … IF NOT EXISTS made each re-emit a silent no-op, so it wasn't data-destructive — just a plan that never went empty.) The fix introspects expression indexes (with a NULL column + an expression flag) and matches them by NAME + uniqueness, since the DB normalises the expression text ((lower("email")) → lower(email)) and it can't round-trip byte-for-byte. Plain-column indexes were never affected. If you see this, update the framework.

Second cause — the same table name in two schemas. If the phantom add-index is for PLAIN-column indexes (often camelCase like "<table>_tenantId_idx") and your DB has the SAME table names in more than one schema — classically a public legacy/migration copy alongside the app's own schema (e.g. voltro) — that was a separate introspection bug (now fixed). The table-list query joined pg_class by NAME, so a name present in both schemas fanned out to two rows → the table was listed twice → its columns and index-columns were accumulated twice in the BUILT snapshot (["tenantId"] became ["tenantId","tenantId"]) → the planner diffed ["tenantId"] != ["tenantId","tenantId"] and re-emitted forever. The raw pg_* catalog looks correct (the duplication is in introspect's built output, not the SQL) — to confirm it's THIS, call the inspect endpoint and look for doubled columns: curl "$API/_voltro/inspect/migrations" | jq '.drift.liveSnapshot.tables[] | select(.name=="<table>") | .indexes'. The fix scopes the table list to current_schema() by OID (+ defensive dedup), so each table is read once. Point DB_SCHEMA / the connection's search_path at your app schema and update the framework.

#duplicate index name '<name>' across tables '<a>' and '<b>'

Index names are unique per schema, not per table, in every dialect. If you gave the SAME explicit name to indexes on two different tables (.index('byStatusStart', …) on both ab_tests and tournaments), boot / db plan now fails loud with this error instead of silently creating only one and re-emitting the rest forever. Fix: rename the collisions to distinct, table-scoped names (abTestsByStatusStart, tournamentsByStatusStart). Auto-named indexes (.index([col]) → <table>_<col>_idx) are table-prefixed and never collide — only hand-picked names can. (If you're updating from an older build that let these through, expect this error on first boot for every pre-existing collision — rename each one it names.)

#auto-named index '<table>_<col>_idx' … exceeds the 63-byte … limit

The framework derives an FK auto-index name from the table + column name (<table>_<col>_idx). On a long junction table that can exceed 63 bytes — and the DB silently truncates index names (postgres → 63 bytes, dropping the _idx suffix), so the declared name (…_idx) never matches the live (truncated) one and db plan re-emits it forever. The explicit-index path was always length-validated; this closes the gap for the auto path — it now hard-fails at boot (same policy as every other identifier: no silent truncation). Two fixes, your choice:

• Add an explicit short name for that FK column — .index('<short>', ['<col>']) — which replaces the auto-index, OR
• Shorten the table / column name.

(Updating from an older build that truncated these? Expect the error on first boot for each one — apply one of the two fixes per index it names.)

#db plan re-emits alter-column-default for a json().default({…}) column

A json() column with an OBJECT default (json<T>().default({ a: 1 })) had two problems on postgres (both now fixed): (1) the default was silently dropped — the DDL emitter only handled scalar defaults, so the column got no default at all (an omitted field inserted NULL, not the object); and (2) even once present, the comparison didn't match — postgres stores a jsonb default in canonical text ('{"a": 1}'::jsonb: spaces after :/, and keys reordered by length/bytes), which never equals the declared JS object's JSON.stringify, so db plan re-emitted alter-column-default every run. The fix emits object defaults as '<json>'::jsonb AND canonicalises both sides (key-sorted, space-free) before comparing. Update the framework.

Cross-dialect: object literal defaults are now emitted to DDL on every dialect, in each one's json idiom — postgres '<json>'::jsonb, mysql/mariadb (CAST('<json>' AS JSON)) (a literal default is rejected on a JSON column), mssql/sqlite a '<json>' string literal — and the comparison canonicalises each engine's introspected form (postgres reorders + spaces; mysql wraps in cast(…); mssql wraps in ('…')). Arrays are unaffected (an array default stays as-is — its json[] vs native array() column is ambiguous). If you need a per-insert dynamic value instead of a fixed literal, use a factory .default(() => ({ … })) (the store applies it at insert).

#MySQL db plan / db apply crashes: Cannot read properties of undefined (reading 'toLowerCase')

MySQL 8 returns information_schema result columns in UPPERCASE (DATA_TYPE, COLUMN_NAME, …) where MariaDB returns lowercase. The introspector read the lowercase fields, so on MySQL the type mapper got an undefined data type and the whole introspect (every db plan / db apply) crashed. Fixed — the introspector now lowercases each information_schema row's keys (no-op on MariaDB). If you hit this on MySQL, update the framework. (MariaDB was never affected, which is why it went unnoticed — the introspect tests run on MariaDB.)

#"no schema files found"

db plan: no schema files found
  hint: looked for *.entity.ts / *.schema.ts / schema.ts
        root: /home/me/myproject/apps/api

What happened: the discovery walker didn't find any schema files under the project root.

Fix:

• Check you're running the command from the right directory (pwd)
• Check your entity files match the convention (apps/api/database/*.entity.ts)
• Run from the project dir, or pass the path as a POSITIONAL arg (voltro db plan ./apps/api) — there is no --root flag; the CLI resolves the root from the first non-flag argument, defaulting to the current working directory

#When the fix hint doesn't match reality

The fix hints come from the planner's classification logic — they should always be actionable. If you see one that doesn't make sense given your code:

1. Check git: were there uncommitted schema changes you forgot about? git status
2. Check the introspect output: curl <app-url>/_voltro/inspect/migrations | jq '.pending' → see the raw plan
3. File an issue with the schema + the inspect JSON + the message

Hint mismatches are bugs in the planner's classification — they're rare but always worth reporting because they're typically reproducible.

#Where to learn more

• Operation classes — the seven classes + per-class examples
• Backfill — SQL vs JS + the dry-run pattern
• Rename and drop — the .renamedFrom() + dropped() lifecycle
• Drift — when the live DB diverged
• Multi-dialect strategy — why MySQL + forward-roll
• Prod pipeline — the deploy-step apply pattern