Time-travel debugger

The time-travel debugger records the app's ChangeEvent stream into a bounded, redacted ring and lets you scrub any table's rows back and forward through it — replaying what the data looked like at any recorded point. Read-only: replay reconstructs a row-set by reversing the recorded changes over the live rows; it never writes.

#Turn it on

Recording is env-gated and off by default in production (volume + PII are the real cost — use OTLP for prod forensics). Set VOLTRO_TIMELINE:

value	records
off (also 0 / false)	nothing
interesting (also on / 1 / true)	user tables only (skips _voltro_* framework noise)
all	every table (except the timeline's own writes)

The default is interesting outside production, off in production — the same environment-aware default the durable trace persistence uses.

Sensitive-looking columns (password / secret / token / api_key / authorization / cookie / ssn / credit_card / cvv / …) are redacted before a row enters the ring — PII never sits in the debug buffer.

#Scrub it

Open the Time-travel panel in the DevTools dashboard (or the cloud dashboard, for a deployed app): pick a table, drag the scrubber across the recorded sequence, and the panel reconstructs that table's rows as of the selected point. The event log shows every recorded change; click one to scrub to just before it. "Jump to now" returns to live. The scrubber uses the dashboard accent and a visible keyboard focus ring; arrow keys change the selected sequence and End returns to the newest position. Sequence and recovery controls wrap on narrow screens.

Protected cells explain why their value is hidden: server-only, encrypted or sensitive. They remain distinct from empty values and ordinary JSON. Event badges retain their operation labels and use theme-aware text colors. Table selection and the scrubber share a compact control area above the rows. Long table lists scroll within that area instead of pushing the data farther down the page; the selected table is exposed to assistive technology.

On narrow screens, choose a table from the compact selector. The sequence and “Jump to now” stay together above the rows; the full event details remain in the event list below.

#Inspect endpoints

Both endpoints use the app's inspect authentication and are available in development, source serve and compiled production builds. The example uses curl and jq; supply your configured inspect authentication when required.

• GET /_voltro/inspect/timeline returns recording state, recorderId, currentSeq and events. Optional filters are table, tenantId, from, to and limit.
• GET /_voltro/inspect/timeline/replay?table=<t>&seq=<n>&recorderId=<id> reconstructs the selected table within that recorder's retained history.

# Use the identity and sequence from the same successful timeline response.
timeline_snapshot=$(curl --fail --silent "http://localhost:4000/_voltro/inspect/timeline?table=todos&limit=50")
timeline_recorder=$(printf '%s' "$timeline_snapshot" | jq -r '.recorderId')
timeline_sequence=$(printf '%s' "$timeline_snapshot" | jq -r '.currentSeq')
curl --fail --get "http://localhost:4000/_voltro/inspect/timeline/replay" --data-urlencode "table=todos" --data-urlencode "seq=$timeline_sequence" --data-urlencode "recorderId=$timeline_recorder"

A missing or replaced recorder identity, an expired position or a change during the replay read returns HTTP 409. Reload the timeline and explicitly select a current position; never reuse an old sequence with a new identity. The dashboard's Jump to now action does this.

#Limits

• Replay is read-only. It never restores or changes application rows.
• History is bounded and process-local. Restarting or invalidating the recorder retires its identity. Choose one replica for historical replay; a sequence from one process is not a position in another process.
• Missing, oversized or coalesced change images cannot reconstruct an exact earlier state; replay rejects history crossing those events.
• Production masks .sensitive(), .encrypted() and .serverOnly() fields before recording and again when returning events or replay rows. Sensitive-looking field names are also redacted. Native SQLite, native local Turso, remote libSQL, embedded libSQL replicas, PostgreSQL, MySQL, MariaDB and SQL Server apply the same schema masking in development because their source journals persist row images. Other development stores keep their existing local schema visibility.
• For stores other than native SQLite, native local Turso, remote libSQL, embedded libSQL replicas, PostgreSQL, MySQL, MariaDB and SQL Server, replay reads all table pages in one store transaction, using the dialect's existing isolation. This does not establish atomicity between database rows and external change delivery. Writes absent from the process change stream remain an open consistency limitation; Timeline is not a complete database audit log. Production recording stays off by default.

Migration tombstones declared with dropped() are not captured columns. They are excluded from journal images and current-row snapshots, including while a former physical column still exists.

#Native SQLite source capture

Replacing the capture schema or privacy contract retires existing recording leases and clears retained capture images in the same migration transaction. If that transaction rolls back, the previous capture state is preserved.

Capture checks the installed trigger definitions before accepting history. Catalog whitespace formatting is ignored; changes to quoted values, identifiers, operators or token boundaries still invalidate the contract.

The source position follows journal entries visible in the same transaction, together with the retention floor. A sequence number reserved by an unfinished or rolled-back write does not advance the reader or a newly activated observer.

Writes without an active recording lease do not retain row images. They change the source identity transactionally instead. If such a write commits after an observer activates, that observer refuses the incomplete history and starts a fresh baseline; a rolled-back write does not invalidate it. Existing capture triggers are refreshed through the normal schema apply/deploy path.

For native SQLite, development schema setup and successful voltro db apply, voltro db deploy, or voltro migrate --create-only prepare the capture schema after the application and runtime storage schemas. Serving does not install or repair capture schema. After a schema-changing step/file migration, snapshot restoration, or garbage collection, run the normal schema apply/deploy path before resuming historical inspection.

Enabled SQLite recording reads a bounded database change journal. Writes through another connection or replica are included while a recording lease is active. Table rows, source position and journal events are read in one transaction; paginated table reads use that same snapshot. Sensitive, encrypted and server-only fields are masked before persistence, including in development. Unsupported or oversized images make that history unavailable rather than guessing earlier values.

The server renews its lease in the background and releases it on shutdown. Expiration, schema incompatibility or a retention gap retires the recorder identity; recovery starts at the current source position and never bridges an unrecorded interval. An unprepared database reports unavailable history instead of silently falling back to process-local changes. This remains a bounded debugger, not an audit log. Stores without source capture retain their existing change-stream consistency limits.

SQLite-compatible capture sessions invalidate their in-memory history before releasing the lease. Lease release is best effort with a one-second interruption deadline; an unavailable database does not hold that cleanup indefinitely. If release cannot be delivered, the existing database-time lease expires normally. Native process cleanup can add a short delay after interruption.

Capture-session database operations have a five-second interruption deadline. Readers also have a seven-second deadline that includes waiting for another session operation. A timeout reports unavailable history; failed source work invalidates the recorder before recovery. Renewal starts after ten seconds and polling after one second, avoiding duplicate work immediately after activation. These deadlines include cooperative interruption; native worker cleanup may add a short delay.

#PostgreSQL source capture

PostgreSQL uses a transactional source journal in the connected database namespace (current_schema()). Development schema setup and successful voltro db apply, voltro db deploy, or voltro migrate --create-only prepare its tables and triggers after application and runtime storage. Source and compiled production serving only read this contract; they do not install or repair it. An unprepared or incompatible source reports unavailable history.

While a recording lease is active, committed writes from other connections and replicas are captured even if LISTEN change notifications are interrupted. Each read uses one repeatable-read snapshot for the source events and all requested table pages. Transaction visibility determines which changes are new: an allocated journal number is not commit order, so a transaction that commits late is not skipped. Sensitive, encrypted and server-only fields are masked before journal persistence, including in development. Unsupported or oversized images make history unavailable instead of allowing an approximate replay.

Each recording process renews a database-time lease in the background and releases it on graceful shutdown. Expired leases, incompatible schemas and retention gaps retire the recorder identity. Collection invalidates readers that still need discarded events; recovery starts from a fresh snapshot without bridging missing history. When a writer sees no active lease, it stops saving row images and records a compact transaction gap instead. An abandoned lease can retain images until it expires. This is bounded debugging history, not a durable audit log or cross-replica sequence.

Recording runs on the app's own connection pool. While nothing has been written since the last capture, each process asks one question per second and captures every ten seconds to renew its lease; a capture is three short transactions, and how many statements they issue does not depend on how many tables are captured. To stop recording entirely, set VOLTRO_TIMELINE=off.

After a schema-changing file or step migration, snapshot restoration or schema garbage collection, use the normal apply/deploy path to prepare capture again. Schema maintenance retires the old contract before changing tables, so historical inspection cannot silently reuse it.

#MySQL and MariaDB source capture

MySQL and MariaDB use a guarded source journal in the connection's selected database. Development schema setup and the normal schema apply/deploy path prepare its capture tables, triggers and generation helper after application and runtime storage. Serving reads that contract without repairing it. Recording does not fall back to process-local changes when the contract is unavailable.

Initial provisioning and repair require a migration identity with the schema DDL/routine privileges and global PROCESS access needed to read InnoDB physical table generations. The generation helper is parameterless and fixed to its own schema. Voltro does not create a privileged account or grant global privileges. An already provisioned, unchanged contract can be reused by a schema-privileged application account without PROCESS; it needs access to the capture tables, trigger metadata and helper routine. This does not replace the database driver's existing CDC permissions, such as binlog replication rights. The recorder needs at least two pool connections for the migration coordinator and its guarded operation. Preparation also follows the runtime runner-storage selection: SQL runner storage needs at least three connections during initial bootstrap. The existing VOLTRO_WORKFLOW_RUNNER_STORAGE=memory option keeps the single-process runner path and its two-connection floor; it does not provide cross-pod workflow handoff.

If optional capture provisioning fails, startup reports a warning and continues. Timeline history remains unavailable until a suitably privileged migration prepares the contract. Application reads and writes remain usable; an unavailable recorder never presents an incomplete local change stream as coherent history. Schema-changing migrations retire the old capture contract before changing application tables. Prepare capture again through the normal apply/deploy path afterward.

While an observer is active, committed writes from other connections and replicas are recorded. Rows and journal events are read from one guarded repeatable-read snapshot, including all requested row pages. Exact visible event identities preserve transactions that commit later with a lower allocated journal number. Sensitive, encrypted and server-only fields are masked before persistence, including in development. Unsupported or oversized images are rejected.

The recorder decodes a complete frame, acknowledges its position and validates retention before publishing it. Expiry, missing capture evidence, physical table replacement such as TRUNCATE, schema changes or interrupted publication retire its identity. Recovery starts at current rows and does not bridge missing history.

Background observation checks for changes every second and renews the observer after ten seconds. Idle checks use private capture storage instead of scanning application tables. Capture windows expire on the database clock; writes without a live observer do not retain row images. Closing the final observer stops capture. The recorder and ordinary migrations use the same configured migration-lock scope, which is separate from the selected MySQL database name.

#Native local Turso source capture

Native local Turso uses a fixed ring of 5,000 capture slots. Development startup and successful voltro db apply, voltro db deploy, or voltro migrate --create-only prepare the capture contract. Source and compiled production serving validate and read it without installing or repairing it.

Readers track each slot's generation in a database snapshot, together with all requested row pages. A lower sequence that commits late is still collected. Overwritten slots, writes outside an active recording lease, an expired lease, or an incompatible contract invalidate history; recovery starts a new baseline. The ring is bounded storage, not a guarantee of the last 5,000 committed changes or an audit log.

Sensitive, encrypted and server-only fields are masked before capture, including in development. The undo ledger's serialized row images are server-only too, so nested application values cannot bypass this masking through that ledger. The undo engine retains its original data and behavior.

This applies to the native, single-process embedded Turso backend. Remote libSQL and embedded-sync clients use the journal contract described below. It does not provide concurrent cross-process access to one native database file.

#Remote libSQL and embedded replicas

Remote libSQL and file-backed embedded replicas use the SQLite journal capture contract, not the native Turso ring. Normal development startup and schema preparation install it; source and compiled serving validate the prepared contract. External primary writes enter the history, and server-only, sensitive and encrypted fields are masked before journal persistence. Embedded snapshots read rows and journal from the same local snapshot; their freshness follows replica synchronization.

An embedded replica owns one write connection. Autocommit statements, write transactions and reserved connections share a scoped permit, so background lease work cannot overlap an active transaction on that connection. Local read snapshots remain independent. Cancelling a waiting writer does not cancel the active transaction. Source interruption or lease loss invalidates the old recorder identity; recovery starts a fresh baseline.

Direct libSQL transaction acquisition and commit/rollback waits support Effect interruption. If acquisition completes after cancellation, the adapter closes the late transaction handle without entering the cancelled body. Cancellation releases local ownership; it does not prove that a remote server has stopped processing a request or rolled back a write. Interrupted writes are not replayed automatically. Remote transaction cleanup waits for in-flight work and the rollback acknowledgement in the background. New transactions wait for that cleanup before opening another server transaction; their acquisition wait remains interruptible. This prevents recovery retries from accumulating behind cancelled transactions. Recovery still depends on the primary becoming reachable; an unavailable history is not a promise of immediate recovery.

#SQL Server source capture

In an atomic SQL Server schema deployment, capture retirement joins the migration transaction: schema changes and capture retirement commit together, and a rollback restores both the previous schema and its capture contract. Standalone file-migration retirement commits before the file body runs. Serving replay frames still require their own independent read transaction.

When capture metadata is missing but Timeline triggers remain on tables resolved by the current store, schema maintenance refuses to continue rather than treating the capture as absent. This also applies when a user default schema falls back to dbo. Triggers on tables shadowed by the default schema are not adopted or removed.

SQL Server uses a migration-owned change journal and persisted activation contract. Normal development/schema preparation installs capture; serving only discovers and reads an existing contract. Database Change Tracking must already be enabled. Preparation requires schema DDL and metadata visibility; serving needs access to the capture tables and object definitions. Voltro does not grant database rights. If optional preparation fails, application startup continues with a warning and Timeline remains unavailable; it does not fall back to an incomplete local stream.

Capture resolves the actual schema of unqualified application tables, including the user's default-schema fallback to dbo. Mixed physical schemas are rejected. Its migration lock uses the ordinary configured lock scope independently of the physical table schema. Observer maintenance requires at least two pool connections.

While a database-clock observer window is live, triggers record committed row changes from other connections. Snapshots read all pages and journal entries in one guarded serializable transaction. Exact decimals and large integers, masking and JSON decoding match capture images. Incomplete or oversized images reject the whole frame. The recorder decodes, acknowledges and checks retention before publishing; failed or interrupted frames retire its identity. History remains bounded and process-local, not a durable audit log or a shared replica sequence.

Schema maintenance retires capture before application DDL. Prepare it again through the normal schema apply/deploy path afterward. Generation, physical catalog and persisted activation checks reject stale history; discovery never silently adopts a replacement generation. A running session can discover later provisioning or a new activation and begin a fresh baseline without bridging the missing interval.

SQL Server schema apply/deploy also prepares the runtime-owned workflow cluster storage before capture activation. Run those commands with the migration identity. A serving identity without DDL rights uses VOLTRO_AUTO_MIGRATE=0 against that prepared database; its ordinary application and workflow storage permissions, including VIEW CHANGE TRACKING on CDC-reader tables, are required in addition to the Timeline grants below. The serving process does not need to create missing cluster tables on its first start.

Observer cancellation waits for an in-flight SQL request to finish before rollback and lease cleanup; snapshot callbacks remain interruptible. This prevents shutdown from leaving an active query on a reused connection.

An unchanged prepared contract can be reused without schema-wide VIEW DEFINITION or DDL rights. The serving identity needs SELECT and object-level VIEW DEFINITION on captured tables and all four private capture tables. Journal retention needs DELETE on _voltro_timeline; observer maintenance needs INSERT, UPDATE and DELETE on _voltro_timeline_leases, and INSERT/DELETE on _voltro_timeline_capture_windows. It does not need to update activation metadata. Initial provisioning or repair still requires the migration identity. Revoking required object-definition visibility makes an existing recorder unavailable.