Governance

@voltro/plugin-governance is the compliance layer: a periodic retention sweep that deletes or anonymises stale rows, GDPR export + erasure across every table that references a subject, a consent ledger, and field-level encryption for .encrypted() columns. It pairs with @voltro/plugin-audit (who/when) and @voltro/plugin-soft-delete (recoverable deletes).

#Wiring

// app.config.ts
import { governancePlugin } from '@voltro/plugin-governance'

export default {
  type: 'api' as const,
  name: 'api',
  plugins: [
    governancePlugin({
      // Retention: delete/anonymise rows past a TTL on a periodic sweep.
      retention: [
        { table: 'events', ttlMs: 90 * 86_400_000 },                                  // delete after 90 days
        { table: 'users', ttlMs: 365 * 86_400_000, action: 'anonymize', anonymizeFields: ['email', 'name'] },
      ],
      // GDPR: DERIVE which tables hold a subject's data from the schema.
      deriveSubjectScopes: { subjectTable: 'users' },
      // …plus anything the schema cannot encode (see below).
      subjectScopes: [
        { table: 'audit_trail', subjectField: 'actorRef' },
      ],
      sweepIntervalMs: 3_600_000,   // default 1h
    }),
  ],
}

#Retention

runRetention walks each policy's table and, for rows older than ttlMs (measured from dateField, default createdAt), either deletes them (action: 'delete', default) or nulls anonymizeFields (action: 'anonymize'). The sweep is armed in bindDataStore and stopped on onDeactivate; trigger it manually via the service's runRetentionNow(). On a multi-replica SQL deployment the sweep is cluster-coordinated — it runs on only ONE replica per tick (via the same claim gate the cron scheduler uses), so N replicas no longer each run the full paged scan; single-process / memory / SQLite deployments run it locally every tick. No configuration — it's automatic.

#GDPR — export + erasure (admin-gated)

Two admin-only routes (guarded by requireScope(ADMIN_SCOPE)):

• governance.export { subjectId } → a portable bundle { [table]: rows[] } of everything belonging to the subject.
• governance.erase { subjectId, mode? } → deletes (or anonymises) the subject across every scope; returns an immutable ErasureLogEntry ({ subjectId, at, mode, affected: [{ table, count }], truncated? }).

The same operations are available in-handler via GovernanceService (exportSubject / eraseSubject).

Erasure runs deepest-first — children before parents — so a real foreign key neither refuses the delete nor cascades through rows the log entry never counted.

#The subject scope is DERIVED from your schema

A hand-written list of "every table holding this person's data" is wrong the day after someone adds a table — and that list is the compliance claim. So deriveSubjectScopes: { subjectTable: 'users' } walks the schema instead: the relations registry plus the reference() column graph, outward from the subject.

That reaches rows a flat { table, subjectField } entry cannot even express — users → posts → comments is two hops, so a comment on the subject's post is in the export without anyone listing comments.

Only CHILD edges are followed — a table holding a reference to the subject's row. Never a parent or lookup edge, and a manyToMany follows the JUNCTION only. Getting that backwards would not be an over-broad export; it would be an erasure that walks from one member into their organisation and deletes everybody else's rows.

declaration	followed?
many(posts, { foreignKey: 'authorId' })	✓ the target carries the FK
one(profile) with the FK on the target	✓ a 1:1 child
one(country, { sourceKey: 'countryId' })	— a lookup this row points AT
manyToMany(orgs, { through: memberships })	✓ memberships only, never orgs
a reference() column with no relations() block	✓ found in the column graph

subjectScopes is still first-class and is unioned on top, never replaced. It is the only way to reach a link the schema does not encode: a subject id in a plain (non-reference()) column, a polymorphic (ownerType, ownerId) pair, an id inside JSON.

#What the derivation cannot see — and says so

A list of reachable tables, printed alone, reads as a completeness claim. So the blind spots ship in the same payload:

const gov = yield* GovernanceService
const { paths, limitations } = gov.subjectGraph()

…and at GET /_voltro/inspect/plugins/governance/subject-graph. limitations names every table nothing links to the subject (unreachable), everything cut by the depth ceiling (depth-truncated, default 4 hops) and everything you excluded. Structurally outside the graph in every case: object storage and uploaded files, external processors, log and metric sinks, backups, and any subject id embedded in JSON or free text.

The plugin also warns at boot if a derived scope reaches one table or fewer — that is what a typo'd subjectTable looks like, and it is otherwise indistinguishable from a working configuration until the first DSAR.

#voltro privacy

voltro privacy scope --subject-table users        # the graph + its blind spots, OFFLINE
voltro privacy scope --json                       # { reachable, unreachable, depthTruncated }
voltro privacy export usr_123 --url https://api.example.com --out bundle.json
voltro privacy erase  usr_123 --url https://api.example.com --confirm

scope needs no database and no running app — schema only, so it belongs in CI and in a PR review, where "does our erasure still reach every table" is a question somebody can still act on.

export / erase deliberately go through the running app's admin-gated governance endpoint rather than opening their own connection. A CLI that erased directly would bypass your configured anonymizeFields and exclusions, bypass the erasure log (which is the compliance artefact, not a nicety), and work against a schema the deployed app may not be running. erase refuses without --confirm, without --url, and without an inspect credential.

#Scale

Every read is WHERE <column> IN (<keys>) on an indexed column, chunked at 500 keys and memoised across paths that share a prefix — not a full table scan per scope. Per table, 50 000 rows is the ceiling; hitting it sets truncated on the erasure-log entry and exits non-zero from the CLI, because a short erasure presented as complete is exactly the failure this is built to prevent.

#Crypto-shredding is NOT supported — and should not be faked

"Erase a subject by destroying their key" needs a key per subject. The shipped cipher is one app-wide passphrase-derived key, so there is nothing subject-shaped to destroy: deleting it would make every subject's .encrypted() columns unreadable, which is an outage, not an erasure. Per-subject shredding needs envelope encryption — a DEK per subject, wrapped by a KEK, with every existing ciphertext re-wrapped — which is a re-architecture of the cipher rather than a mode of eraseSubject.

What makes its absence cost less than it sounds: .encrypted() columns are decrypted transparently on read, so delete removes the ciphertext row and anonymize overwrites the ciphertext with a null. Both erase the data itself rather than the key guarding it. Crypto-shredding is an optimisation for erasure at rest across backups; it is not the only route to Art. 17.

#Consent ledger

governance.consent { purpose, granted } records the calling subject's decision; governance.hasConsent { purpose } reads the latest (latest-write-wins per (subject, purpose)).

The default store is in-memory — per-process + restart-cleared, so a consent recorded on one replica is invisible to others (the plugin warns at boot when it runs under this default). For production pass consent: 'datastore' for the shipped durable, cross-instance store: it contributes a _voltro_consent table (via extendSchema, under the store:write permission the plugin already declares) and appends one row per decision, so a consent recorded on any replica is visible everywhere and the full history persists for audit. Or pass a custom ConsentStore.

governancePlugin({ consent: 'datastore' })   // durable + cross-instance

import { GovernanceService } from '@voltro/plugin-governance'

export default (input, ctx) => Effect.gen(function* () {
  const gov = yield* GovernanceService
  if (!(yield* Effect.promise(() => gov.hasConsent(ctx.request.subject.id, 'marketing')))) return { skipped: true }
  // … send the marketing email …
})

#Field-level encryption

Flag a column .encrypted() in the schema and fieldEncryption: true registers an AES-256-GCM cipher that the store middleware applies transparently — writes encrypt, reads decrypt, handlers always see plaintext, the column stores an opaque enc:v1:… string on every dialect.

// schema — database/patients.entity.ts
export const patients = table('patients', {
  id:   id(),
  name: text(),
  ssn:  text().encrypted(),       // AES-256-GCM at rest
  notes: json().encrypted(),      // any column type — JSON-encoded then encrypted
})

// app.config.ts
governancePlugin({ fieldEncryption: true })   // key from secret VOLTRO_FIELD_ENCRYPTION_KEY
// or: fieldEncryption: { secretKey: 'MY_KEY_NAME' }

The key resolves through the Secrets-Resolver — pass a 64-hex-char string for raw key bytes, or any passphrase (scrypt-derived). Rules:

• Lose the key, lose the data. GCM authentication fails closed — a wrong/tampered value throws on read, never silently corrupts. The failure is a typed FieldDecryptionError naming the table.column (not a raw malformed ciphertext with no context, and never the ciphertext itself), so you can see WHICH column and Effect.catchTag it server-side. If it reaches the wire UNCAUGHT, it collapses to a generic InternalError for the client (the table.column and cause stay in the server log) — never an ExitEncoded schema dump.
• An .encrypted() column can't be filtered or sorted by plaintext in SQL (it's ciphertext on disk). Encrypt only what you read back whole — PII, tokens, free-text notes.
• Boot fails loud if an .encrypted() column exists but no cipher is registered (enable fieldEncryption), or if fieldEncryption is on but the key can't resolve.
• Reads of rows written before encryption was enabled pass through untouched (only enc:v1:… values are decrypted), so you can turn it on incrementally.

#Restoring a snapshot encrypted under a different key

Importing a prod/staging DB dump into a dev DB brings ciphertext bound to the source key — your dev VOLTRO_FIELD_ENCRYPTION_KEY can't read it, and by default one such row throws a FieldDecryptionError that fails the whole read (including its readable siblings). For dev/migration only, set:

VOLTRO_FIELD_DECRYPT_ON_ERROR=null

Now an undecryptable column degrades to null with one deduped warning per table.column (scope store.fieldEncryption) instead of nuking the read — the row's other columns still decrypt. This is the difference between "re-enter your Jira PAT" and "the whole section 500s". Never set it in production, where a key mismatch must fail loud. A cutover tool that writes rows into .encrypted() columns should NULL those columns on import — the source ciphertext is useless without the source key, and users re-enter the credential so it re-encrypts under the local one.

#Raw-SQL escape hatch — encryptField / decryptField

Transparent encryption runs INSIDE the ctx.store middleware, so a code path that reaches the DB by raw SQL — an auth strategy writing/reading a session token with no store handle, a one-off backfill — bypasses it. For those, @voltro/runtime exposes the SAME registered cipher standalone:

import { encryptField, decryptField } from '@voltro/runtime'

// write path (raw SQL): encrypt by hand
await sql`INSERT INTO sessions (id, token) VALUES (${id}, ${encryptField(pat)})`
// read path (raw SQL): decrypt by hand
const token = decryptField(row.token)   // a non-ciphertext value returns unchanged

decryptField returns a non-enc:v1:… value unchanged, so you can switch a raw-SQL path to encryption while pre-existing plaintext rows keep working until they're re-written. Both throw a clear error if no cipher is registered (enable fieldEncryption). Encryption stops being all-or-nothing tied to going through ctx.store.

One encoding, both directions. These helpers and the store middleware write the same thing, so a value written by encryptField reads through ctx.store and vice versa. That was not always true: the store encoded JSON and the helpers did not, both under the same enc:v1: envelope, and a value written by one and read by the other either threw blaming the KEY — while the key was fine — or came back with the JSON quotes still on it and raised nothing.

Rows written under the OLD encoding still read — nothing has to be rewritten. Both forms are resolved on read, deterministically: after decrypting, a value that does not parse as JSON is the raw form, one that parses to a string is the JSON form, and one that parses to a non-string is decided by the column's declared type (a text column cannot hold a number, so 12345 is a raw string that parsed by accident). This matters because the old form is already on staging and production disks, and a fix that needs the data rewritten before the app works is an outage with a migration attached.

Normalising is optional hygiene, and voltro db encrypt-column does it: it re-encodes rows in the old form as it goes and reports them separately from the ones it encrypts. It skips anything ambiguous and anything it cannot decrypt.

Backups are unaffected either way. voltro data export reads through the raw store, so an encrypted column travels as ciphertext in whichever encoding it holds, and comes back unchanged.

#Dashboard panel

Both dashboards ship a Governance panel (api apps): retention-policy status + field-encryption state + last-sweep results (with a Run sweep now button), a GDPR runner (enter a subject id → Export / Erase delete / Erase anonymize), a consent-ledger lookup, and the erasure log. Write-actions gate on the canRunGovernance capability. Backed by /_voltro/inspect/plugins/governance/{status,erasures,consent,export,erase,sweep}.

#Permissions

store:write (retention sweep + GDPR erasure mutate rows; field encryption rides the store middleware) + inspect:read (dashboard panel).