Storage

@voltro/plugin-storage is file storage behind one StorageService. Wire a provider in app.config.ts; consume it in handlers and actions via yield* StorageService. The bytes live in the provider (S3-compatible bucket, filesystem, or in-memory); metadata rows live in _voltro_storage_refs + _voltro_storage_grants (both auto-migrated on a SQL store).

Every object is public or private:

• Public — cacheable + no auth. With cdnBaseUrl (or STORAGE_CDN_URL) set to a CDN / publicly-readable bucket endpoint, the serve route 302s straight there and the app never touches the read path (edge-cacheable). WITHOUT cdnBaseUrl, the serve route streams the bytes through the app from the (private) bucket with a 1-year immutable cache — still browser-cacheable, bucket stays private. A bare S3/MinIO bucket is private by default, so the framework never 302s to a raw bucket URL unless you've declared a public base via cdnBaseUrl (otherwise it would land on a 403).
• Private (the default — secure by default) — gated by an access policy plus per-object grants; delivered via a presigned URL or a short-lived signed grant token.

#Quick start

// app.config.ts
import { storagePlugin } from '@voltro/plugin-storage'

export default {
  type: 'api' as const,
  name: 'myApi',
  store: 'postgres' as const,
  plugins: [
    // dev: zero-config in-memory. prod: swap the provider (see below).
    storagePlugin({ provider: 'memory' }),
  ],
}

// actions/uploadAvatar.action.server.ts (+ matching .action.ts)
import { StorageService } from '@voltro/plugin-storage'
import { Effect } from 'effect'

export default (input: { bytes: Uint8Array; contentType: string }, ctx) =>
  Effect.gen(function* () {
    const storage = yield* StorageService
    const ref = yield* storage.put({
      bytes: input.bytes,
      contentType: input.contentType,
      visibility: 'public',                // public avatar
      ownerId: ctx.request.subject.id,
      key: `avatars/${ctx.request.subject.id}/avatar.png`,
    })
    // For a public object this is the CDN/bucket URL (no app round-trip).
    return { url: yield* storage.getUrl(ref.id) }
  })

The ref id is a file_… TypeID — URL-safe and unguessable.

#Uploading files — useUpload() (client)

The one-liner. Works on every provider (filesystem/dev, s3/r2/prod), cross-origin, no base64:

import { useUpload } from '@voltro/client'

function AvatarUpload() {
  const { upload, progress, status, cancel } = useUpload('myApi')
  return (
    <input type="file" onChange={async (e) => {
      const file = e.target.files?.[0]
      if (!file) return
      const asset = await upload(file, { visibility: 'public', folder: 'avatars' })
      // asset: { id, url, name, mime, size, width?, height? }
    }} />
  )
}

useUpload(apiName) returns { upload, uploadMany, progress (0..1), status, error, cancel, reset }. Under the hood it calls the authenticated storage.mintUploadTicket rpc (which runs your auth chain and returns a short-lived signed URL), then POSTs the file's raw bytes via XHR with real per-byte progress — binary, so a 50 MB video never bloats the rpc payload. The stored ref row appears live in any useSubscription over that table (normal reactivity). For an instant preview, pair with URL.createObjectURL(file) and swap to asset.url on completion.

Drag-drop + paste come for free. The handle also returns onDrop, onPaste, and a dropzoneProps you spread onto any element for a zero-boilerplate dropzone:

const { dropzoneProps, onPaste, uploadMany } = useUpload('myApi')

<div {...dropzoneProps}>Drop files here</div>              // drop → uploadMany
<textarea onPaste={(e) => onPaste(e, { folder: 'notes' })} />  // paste a screenshot

onDrop(event, opts?) / onPaste(event, opts?) pull the files off the event and call uploadMany (bounded concurrency); both resolve [] when the event carries no files. dropzoneProps also prevents the browser's default "navigate to the dropped file" behavior.

Server-side, the same path is the mintUploadTicket action + the POST /_voltro/storage/upload route (ticket-verified, no auth re-run). The storage.upload base64 action also exists for tiny files.

#Transports — prefer

through-app (the default) works everywhere and needs nothing extra. Two opt-in transports handle large media:

useUpload('myApi').upload(file, { prefer: 'presign' })    // offload the upload leg
useUpload('myApi').upload(file, { prefer: 'resumable' })  // survive dropped connections
useUpload('myApi').upload(file, { prefer: 'multipart' })  // multi-GB, direct to bucket

• presign — on a presigning provider (s3/minio) the browser PUTs straight to the bucket (offloading the client→server leg), then the server fetches the bytes back to scan + checksum + derive and register a non-degraded ref (storage.mintPresignedUpload → direct PUT → storage.finalizeUpload). On a non-presigning provider it transparently falls back to through-app. A virus hit fails closed (temp bytes deleted, nothing registered) — so presigned uploads are scanned, unlike a raw bucket PUT.
• resumable — splits the file into chunks POSTed one at a time to /_voltro/storage/upload/resumable (storage.beginResumableUpload); each chunk is retried independently and is idempotent by index, so a dropped connection resumes instead of restarting. Works on every provider (chunks are stored, then assembled through the same scan pipeline). Tune with chunkSize. Finalization checks the sum of stored chunk sizes against limits.maxBytes before reading any body, then checks actual bytes incrementally as it assembles them. Filesystem, S3/MinIO and Azure use streaming reads; memory/database and custom providers without getStream still buffer one chunk per read. Stale size metadata cannot bypass the actual-byte check, but that fallback can allocate an oversized chunk before rejecting it. The assembly buffer stays within the configured limit; this is not a process-memory cap (provider buffers, buffer growth and the scan/transform pipeline need additional memory). Without limits.maxBytes, no application byte limit is imposed. Constraint rejection attempts to delete stored chunks; provider read failures retain them for retry.
• multipart — for multi-GB media: each part is PUT directly to the bucket via a presigned URL (storage.beginMultipartUpload → storage.signMultipartPart per part → storage.completeMultipartUpload), so the bytes never touch the app — offloaded and resumable (a failed part is re-signed + re-sent). s3/minio only; falls back to resumable elsewhere. Needs the bucket CORS to expose the ETag header (voltro storage cors prints the rule). Tradeoff: completing registers the ref straight from the bucket object (size via HEAD, etag as checksum) — it does not fetch the bytes back, so a multipart upload is not scanned or derived inline (that would mean pulling GB through the app). Scan multi-GB out of band (a bucket-event job) and note checksum is the S3 etag, not a sha-256.

#Media derivatives + rich metadata

put() (and every upload) auto-extracts, for raster images (via optional sharp, graceful when absent):

• width / height (pixels) — the fields your app used to leave null.
• placeholder — a tiny data:image/webp;base64,… LQIP (drops into <img src> / a CSS background; no decoder library).

Opt into storagePlugin({ normalizeImages: true }) to apply EXIF orientation and strip ALL metadata (GPS/camera), re-encoding — the normalized bytes are what get checksummed + stored.

Uploads also carry app metadata onto the ref — folder, tags, alt, caption — so the ref row is rich enough that apps no longer need a parallel assets table.

For video/audio, wire a videoProbe hook — the core bundles no transcoder (ffmpeg is a large binary + licensing), so you plug in ffprobe / a cloud API and it runs in put() for video/audio content types:

storagePlugin({
  videoProbe: async ({ bytes, contentType }) => {
    const { duration, width, height, posterDataUri } = await probeWithFfprobe(bytes)
    return { duration, width, height, poster: posterDataUri } // → ref.duration + ref.placeholder
  },
})

Best-effort: a null / throw / failed probe never blocks the upload. An explicit duration passed to put() wins over the probe.

#Transcoding video — renditions

Produce playable variants (720p mp4, webm, a poster…) from an uploaded video. The first-party ffmpegTranscoder spawns the ffmpeg binary (no npm dep — like clamavScanner talks to a daemon), so install ffmpeg on the host (or pass ffmpegPath); absent it, transcoding is skipped and the upload still succeeds.

import { storagePlugin, ffmpegTranscoder } from '@voltro/plugin-storage'

storagePlugin({
  transcode: ffmpegTranscoder({
    renditions: [
      { kind: 'rendition', label: '720p', contentType: 'video/mp4', ext: 'mp4',
        args: ['-vf', 'scale=-2:720', '-c:v', 'libx264', '-crf', '23', '-c:a', 'aac', '-movflags', '+faststart'] },
      { kind: 'poster', label: 'poster', contentType: 'image/jpeg', ext: 'jpg',
        args: ['-frames:v', '1', '-vf', 'thumbnail'] },
    ],
  }),
})

Each rendition is stored as a normal storage ref linked to the original (derivedFrom = the parent id, kind = 'rendition'/'poster', caption = the label) — so renditions get serving, grants, transforms and GC for free, and inherit the original's tenant + visibility.

• Runs out of band. Transcoding forks in the background after the upload returns (never blocks the client). It's ON by default once a transcode is configured — set autoTranscode: false to only run it explicitly.
• storage.renditions(refId) — list the derivatives of an asset.
• Bring your own transcode (a cloud video API, HLS packager) by implementing the Transcoder interface — ffmpegTranscoder is just one implementation.

#Durability — the auto-fork is best-effort

The background fork is fire-and-forget: if the process crashes or is redeployed mid-transcode, that rendition is simply never produced (the original upload is unaffected). For long or business-critical transcodes, turn the auto-fork off and drive storage.transcode(refId) from a durable workflow — you get retries, backoff, and at-least-once execution across restarts:

// 1. Disable the fire-and-forget fork:
storagePlugin({ transcode: ffmpegTranscoder({ renditions }), autoTranscode: false })

// 2. Kick a durable workflow when a video ref is created (e.g. from the upload
//    mutation or a reactive hook), and transcode inside it — retried until it
//    succeeds, surviving restarts:
export const transcodeVideo = workflow('transcodeVideo', (refId: string) =>
  Effect.gen(function* () {
    const storage = yield* StorageService
    const renditions = yield* storage.transcode(refId)   // retried by the workflow engine
    return { count: renditions.length }
  }),
)

So: convenient by default (auto-fork), durable when you need it (workflow + the explicit primitive). The two coexist — same transcode() call underneath.

ffmpeg transcode (re-encoding) is opt-in as above. On-the-fly image resizing stays separate (query params on the serve URL, powered by sharp).

When a requested resize or format change cannot happen, the serve route answers with the original — and says so: x-voltro-transform: original; reason=not-an-image | no-sharp | failed.

#Binding an entity to an asset — assetRef()

import { table, id } from '@voltro/database'
import { assetRef } from '@voltro/plugin-storage'

const employees = table('employees', {
  id: id(),
  avatar: assetRef(),          // stores a _voltro_storage_refs id
})

assetRef() is a real FK to _voltro_storage_refs.id (nullable, onDelete: 'setNull' — deleting the blob clears the link) by default; pass { fk: false } for a plain typed id column marked as external schema metadata, { nullable: false } to require it, or { onDelete: 'cascade' } to change the FK semantics. The external marker keeps schema audits from proposing a local FK but changes neither storage nor queries. Resolve the linked asset with the storage service (head / mintUrl).

For the one-column shorthand there's a withStorage() mixin (applied via .with, the same convention as tenant() / audit()), which adds a single asset column:

import { withStorage } from '@voltro/plugin-storage'

const posts = table('posts', { id: id() }).with(withStorage())   // → posts.asset

Prefer assetRef() inline when you want a custom column name (avatar, cover) or several asset columns.

#Swappable engine

The provider is the swap point; the rest of your code is provider-agnostic.

storagePlugin({ provider: 'memory' })                                   // dev / tests
storagePlugin({ provider: 'database' })                                 // bytes in your DB — zero extra infra
storagePlugin({ provider: 'filesystem', root: '.voltro-storage' })      // on-disk dev
storagePlugin({ provider: 's3', bucket: 'prod', region: 'eu-central-1',
                cdnBaseUrl: 'https://cdn.example.com' })                 // AWS S3 + CDN
storagePlugin({ provider: 'minio', bucket: 'prod',
                endpoint: 'http://localhost:9000' })                     // MinIO
storagePlugin({ provider: 'azure', bucket: 'prod',
                accountName: 'acct', accountKey: process.env.AZURE_STORAGE_KEY }) // Azure Blob
// Cloudflare R2 / GCS / Backblaze B2 / Wasabi — any S3-compatible bucket via `endpoint`.
storagePlugin({ provider: 's3', bucket: 'prod',
                endpoint: 'https://<acct>.r2.cloudflarestorage.com',
                cdnBaseUrl: 'https://files.example.com', publicAcl: false })

Two providers stand apart from the S3-compatible family:

• azure — Azure Blob Storage (not S3-compatible). Presigned URLs are SAS tokens; needs accountName + accountKey (or AZURE_STORAGE_ACCOUNT / AZURE_STORAGE_KEY), or a connectionString. Requires the optional @azure/storage-blob dependency.
• database — stores the blob bytes IN the app's database (_voltro_storage_blobs, a bytes() / BYTEA column) via the same DataStore. Zero extra infra — no bucket, survives restarts (unlike memory). Served through the app query (no CDN-direct). Use for small files; large blobs belong in object storage.

The filesystem provider writes blobs atomically — bytes go to a unique temp file in the target directory, then rename() over the final path (POSIX-atomic). A reader never sees a half-written file and two concurrent writers of the same key can't interleave into a corrupt blob. That makes it safe on a shared ReadWriteMany volume (e.g. CephFS) with multiple app replicas — point STORAGE_ROOT at the mount. (A ReadWriteOnce block volume mounts on one pod only, so it can't back more than one replica.)

Options resolve from env when omitted: STORAGE_PROVIDER, S3_BUCKET / STORAGE_BUCKET, S3_REGION / AWS_REGION, S3_ENDPOINT / MINIO_ENDPOINT, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, STORAGE_ROOT, STORAGE_CDN_URL, S3_FORCE_PATH_STYLE=1. Pass a StorageProvider object to bring your own backend. publicAcl: false skips per-object public-read ACLs (use a bucket policy + cdnBaseUrl instead — required for R2).

#Public objects — minimal middleware

A visibility: 'public' object is readable with no auth. When cdnBaseUrl is set, getUrl(id) returns ${cdnBaseUrl}/${key} and the serve route 302s there — reads never hit the app. WITHOUT cdnBaseUrl (a bare S3/MinIO bucket, which is private by default — and on filesystem / memory), the framework does NOT hand out a raw bucket URL (it would 403); the serve route streams the bytes through the app instead, with cache-control: public, max-age=31536000, immutable. So public objects are always cacheable; cdnBaseUrl is what moves the read path off the app and onto the edge.

#Private objects — the access model

Access is the union of two layers; a request is allowed if either grants it.

#1. Inline access policy

put({ access }) takes an OR-of-rules array. A rule GRANTS when EVERY condition it declares is satisfied (AND); access is allowed if ANY rule grants. The owner (subject.id === ref.ownerId) is always allowed.

yield* storage.put({
  bytes, contentType: 'application/pdf',
  ownerId: ctx.request.subject.id,
  password: 'optional-file-password',     // hashed; gates a { password: true } rule
  access: [
    { roles: ['admin'] },                 // admins, OR
    { tenant: true },                     // any same-tenant subject, OR
    { groups: ['finance'], scopes: ['files:read'] }, // finance AND files:read
  ],
})

Rule conditions: owner, roles[], groups[], scopes[], tenant, apiKey, password, guard (a named custom guard). Roles/groups come from subject.metadata.roles / .groups by default — override the resolution:

storagePlugin({
  access: {
    resolveGroups: (subject) => lookupGroups(subject.id),   // sync | Promise
    guards: { vip: ({ subject, ref }) => subject.metadata?.tier === 'vip' },
  },
})

#2. Per-object grants

Explicitly share one object with a user / group / api-key, optionally expiring:

yield* storage.grant({ refId, principalType: 'user', principalId: 'user_123', permission: 'read' })
yield* storage.grant({ refId, principalType: 'group', principalId: 'team-a', expiresAt: in7Days })
yield* storage.revoke(grantId)
const grants = yield* storage.listGrants(refId)

Grants persist in _voltro_storage_grants and are managed from the dashboard's Storage tab.

#Delivering a private file

// Access-checked. Returns a presigned URL (s3/minio) OR an app serve URL
// carrying a short-lived signed grant token (filesystem/memory). Fails with
// StorageAccessDenied (403) if the subject isn't allowed.
const url = yield* storage.mintUrl(fileId, ctx.request.subject, { password })

For large uploads, skip the wire and presign a direct-to-bucket PUT (s3/minio): storage.mintUploadUrl({ key, contentType }).

#Making an object public or private later

An object's visibility is set at put and can change afterwards — an image that goes public when its page is published, and private again when the page is taken down:

yield* storage.setVisibility(fileId, 'public', { tenantId })   // readable by anyone
yield* storage.setVisibility(fileId, 'private', { tenantId })  // access-checked again

The tenant is checked like delete. On database, filesystem and memory the ref decides every read. On s3/minio with per-object ACLs (publicAcl) the object follows its refs: identical bytes share one object, so it stays publicly readable while any ref to it is public. Where objects are served from a public base (cdnBaseUrl) a public object cannot be taken back there, and setVisibility(…, 'private') fails with StorageRejected (reason visibility-unsupported) instead of reporting a private object that is not; so does a custom ref store without setVisibility. A response served while the object was public may remain in browser and CDN caches.

signedUrl(id, { expiresInSec, tenantId }) returns a short-lived URL that delivers the bytes without an access check — for server code that already decided the reader may see them (a shared preview whose link it verified): the public URL of a public object, else a presigned GET (s3/minio) or the serve URL with a signed grant token. Never hand one to a caller whose access you have not checked.

Uploads from the browser (useUpload) are stored private unless they ask for public, and a private upload is readable by its uploader only. The app decides who else may read them — never the client:

storagePlugin({ uploadAccess: [{ tenant: true }] }) // everyone in the uploader's tenant

ingestUrl(url, { visibility, access }) takes the same policy for an object fetched from a URL.

#Typed RPC queries (client-facing)

The plugin ships queries your client can call with no extra wiring — and their errors (StorageError, StorageAccessDenied) are merged into every procedure's wire-error union so they decode typed on the client:

storage.upload, storage.mintUploadTicket, storage.mintUrl, storage.mintUploadUrl, storage.mintPresignedUpload, storage.finalizeUpload, storage.beginResumableUpload, storage.beginMultipartUpload, storage.signMultipartPart, storage.completeMultipartUpload, storage.abortMultipartUpload, storage.ingestUrl, storage.share, storage.revoke, storage.listGrants, storage.listRefs. (The upload hook picks the right ones per prefer; you rarely call them directly.)

Every route declares its access decision. The self-service upload/read flow is openAccess — what bounds it is structural (mints bind the caller's subject + tenant into a signed ticket, finalize/complete refuse another subject's token, reads go through the per-object access policy + grants, the grant RPCs enforce owner-or-admin in-handler). Three routes require a scope instead: storage.mintUploadUrl (the raw bring-your-own-key PUT, which bypasses limits + scan) and storage.ingestUrl (the server fetches a caller-supplied URL — an SSRF surface) need storage:manage; storage.listRefs (the tenant-WIDE media library index) needs storage:browse. Grant them via an rbac role or resolveScopes; admin:full passes.

#The grant RPCs are owner-only

storage.share, storage.revoke and storage.listGrants require that the caller OWNS the object (ref.ownerId === subject.id), or carries admin:full.

They used to require nothing. Each took an object id straight off the wire and handed it to a service method that performs no authorization — correct for a trusted server-side API — with nothing in between. Any authenticated caller could grant themselves read or write on any object in the installation, revoke anyone else's grants, and enumerate who an object is shared with.

A missing object and an unowned object both report the same 403; a 404 would let an unauthorized caller probe which ids exist.

If a server-side flow legitimately shares on someone else's behalf, call the SERVICE (yield* StorageService → .grant(...)) rather than the RPC — the service is deliberately unrestricted, and the gate belongs at the wire. For an admin surface, give the subject admin:full (see auth.resolveScopes for lifting a role into scopes).

#StorageService API

Method	Returns
put(input)	StoredRef — a StorageRef { id, tenantId, ownerId, bucket, key, contentType, size, checksum, visibility, accessPolicy, createdAt } plus this call's outcome ('created' | 'resumed' | 'joined'); with operationId the put is resumable, with claim the caller holds it (below)
get(id, opts?)	{ bytes, ref } (tenant-guarded when opts.tenantId set)
getUrl(id, opts?)	direct/CDN URL (public) or presigned/serve URL (private)
mintUrl(id, subject, opts?)	access-checked delivery URL
mintUploadUrl(input)	presigned PUT (s3/minio)
head(id, opts?)	StorageRef | null
getRange(id, { start, endInclusive }, opts?)	{ bytes, ref, totalSize } — a byte slice (backs Range serving)
delete(id, opts?)	refcount-aware — dedup'd siblings survive; durable for a ref written with an operationId; with opts.claim releases that caller's claim only (below)
listRefs(input?)	{ refs, nextOffset } — browse/search the ref index
checkAccess(ref, subject, opts?)	boolean
grant / revoke / listGrants	manage per-object grants

Failures are a typed StorageError (the transient flag drives retry — default 3 attempts) or StorageAccessDenied (403).

#Resumable puts — operationId

A plain put is a fresh object every time it runs. Retried by a job runner, or completed by two replicas at once, that is two references on the same content-addressed key — and the loser's reference outlives the winner's delete. Name the operation and it is one object:

import { Effect } from 'effect'
import { StorageService } from '@voltro/plugin-storage'

export const storeRendering = (jobId: string, bytes: Uint8Array) =>
  Effect.gen(function* () {
    const storage = yield* StorageService
    // Retried by the job runner, run by two replicas — one object either way.
    return yield* storage.put({ bytes, contentType: 'image/png', tenantId: 'acme', operationId: jobId })
  })

What the id buys, in order:

• One reservation before any byte moves. The operation's identity and its quota are written in one transaction; the provider write comes after. A repeat with the same id finds the reservation and either returns the finished ref or completes the half-done write — no second reference, no second quota charge. The two replicas above agree on one ref id and one key.
• Different inputs under the same id are refused (StorageRejected, 'operation-identity-reused'): the identity binds the content checksum and the metadata that shape the object, so a retry cannot quietly store something else under a finished operation's name.
• A durable delete. Deleting such a ref removes its metadata, grants and quota in one transaction and retains the physical key until the provider acknowledges the byte delete. A delete that failed at the provider can be repeated by id; a put of the same content that arrives after the delete has already counted zero gets a new physical key, so its bytes are never taken by the delete that is still in flight. Repeating a finished delete is a no-op.

The id is any stable string the caller owns — a job id, a message id, a webhook delivery id. It needs the app's data store: with the in-memory ref store (no database) an operationId put is refused ('operation-store-unbound') rather than silently made non-resumable.

#Shared objects — claim

One ref shared by two callers has a cleanup problem: a replica that fails after its put and removes "what it put" deletes the file the other replica already stored in its result. Without cleanup, every job that finally fails leaves a file behind. Neither is what you want, and put's outcome cannot decide it — "only clean up what I created" still deletes the joiner's kept file whenever the creator is the one that fails.

A claim is this caller's hold on the operation's object: any stable string the caller owns, an attempt id or a replica id. Each caller that receives the ref holds its own claim; delete(id, { claim }) releases that claim only, and the object — bytes, ref, grants, quota — goes with the last one:

import { Effect } from 'effect'
import { StorageService } from '@voltro/plugin-storage'

// Two replicas finish the same job; each keeps its own claim on the shared object.
export const storeRendering = (jobId: string, attemptId: string, bytes: Uint8Array) =>
  Effect.gen(function* () {
    const storage = yield* StorageService
    const stored = yield* storage.put({ bytes, contentType: 'image/png', tenantId: 'acme', operationId: jobId, claim: attemptId })
    // stored.outcome: 'created' | 'resumed' | 'joined' — informational
    return stored
  })

// A replica that fails later releases ITS claim only. The object goes with the last one.
export const discardRendering = (refId: string, attemptId: string) =>
  Effect.gen(function* () {
    const storage = yield* StorageService
    yield* storage.delete(refId, { tenantId: 'acme', claim: attemptId })
  })

• A claim is registered when the caller receives the ref, never earlier: a caller whose provider write fails holds nothing. The same claim repeated is one claim, and releasing it twice is a no-op — a retried cleanup after a lost acknowledgement is safe.
• A delete that names no claim is the whole object, however many claims it holds. That is the normal lifecycle delete — when the job's result is deleted, its file goes with it — and the admin delete from the dashboard.
• Fail-closed on the wrong side. A claim without an operationId is refused before any byte moves (StorageRejected, 'claim-without-operation'). A claim released on a ref that holds no claims at all, or on a ref not written through an operation, is a StorageError with status 409 — nothing is deleted.
• outcome is informational. 'created' reserved the operation, 'resumed' found it half-done and finished the write, 'joined' found it finished and took its ref. It is a fact about the call and is never stored: a ref read back does not carry it.

A crashed caller never releases its claim; the whole-object delete of the normal lifecycle clears it. Claims need the same bound data store as the operation itself.

#Browse / search files — listRefs()

The folder, tags, and ownerId you set on each put() are queryable first-class — a media library or file manager never has to touch _voltro_storage_refs by hand. listRefs() filters by folder prefix (a path descendant match), all-of tags (AND), and ownerId, newest-first, with offset paging:

import { StorageService } from '@voltro/plugin-storage'
import { Effect } from 'effect'

const browse = Effect.gen(function* () {
  const storage = yield* StorageService
  // Page 1: everything under `photos/` (incl. `photos/2026/…`) tagged `hero`.
  const page = yield* storage.listRefs({
    tenantId: 'org_123',
    folder: 'photos',
    tags: ['hero'],
    ownerId: 'user_42',
    limit: 24,
    offset: 0,
  })
  // page.refs: ReadonlyArray<StorageRef>, newest first.
  // page.nextOffset: number | null — pass back as `offset` for the next page,
  // or null when this was the last page.
  return page
})

• folder matches the value exactly OR as a /-delimited ancestor (photos ⇒ photos, photos/2026; never photos-archive). A trailing slash is ignored.
• tags requires ALL listed tags on the ref (AND).
• ownerId / tenantId scope to one owner / tenant (tenantId: null is the system/global partition; omit tenantId only for a trusted admin read that spans tenants).
• Transcode derivatives (renditions / posters) are excluded by default; pass includeDerived: true to include them.
• Paging: limit is clamped to [1, 500] (default 50); nextOffset is non-null while another page follows.

listRefs filters the index — it does not run the per-object access policy (it delivers no bytes). Scope it to the caller's tenantId / ownerId for an end-user surface, and deliver any listed private object through mintUrl. The storage.listRefs rpc query takes the tenant from the caller's subject server-side, so a client can only browse its own tenant.

Reach the service with yield* StorageService inside an Effect handler — the plugin provides it as the handler base layer. The service methods are Effect-returning (storage.put(...) is an Effect<StorageRef>); from a plain async handler, run them with Effect.runPromise, or write the handler in Effect form.

#Serve endpoint

GET /_voltro/storage/:id (mounted pre-auth):

• public → 302 to the CDN/bucket URL when cdnBaseUrl is set; otherwise serve the bytes with cache-control: public, max-age=31536000, immutable (no auth either way).
• private → checks a ?t=<grant-token> (minted by mintUrl) OR the session cookie + access policy; then 403, a presigned 302, or a served body with cache-control: private, no-store.

#HTTP Range — 206 Partial Content (video seeking, bounded delivery)

When the object is served through the app (a non-presigning provider, or a private object without a bucket presign), the serve route honours an HTTP Range: request header so a <video>/<audio> element can seek and a client can resume — only the requested slice leaves the backend (via the provider's getRange; memory/database fall back to a buffered slice):

• A full GET responds 200 with Accept-Ranges: bytes.
• A satisfiable Range: bytes=<start>-<end> responds 206 Partial Content with Content-Range: bytes <start>-<end>/<total> and just that slice. Open- ended (bytes=N-) reads to EOF; a suffix (bytes=-N) reads the last N bytes.
• A range entirely past the object → 416 Range Not Satisfiable with Content-Range: bytes */<total>.
• A malformed Range header falls back to the full 200 (per RFC 7233).

For presigning providers (s3/minio) a private object 302-redirects to the bucket, which serves Range natively — the client re-issues its Range against the presigned URL, so seeking works without the bytes ever passing through the app. The Range/Content-Range/Accept-Ranges headers are added to the serve route's CORS allow/expose lists so a cross-origin fetch() can drive them.

#Cross-origin: absolute URLs + CORS

When the api is a different origin than your web app (the common deploy — api.example.com vs your web origin), the serve URL must be absolute or a browser <img> / a server-to-server AI-gateway fetch can't resolve it. Set the api's public base and getUrl/mintUrl/the serve route emit absolute URLs:

VOLTRO_PUBLIC_URL=https://api.example.com     # or storagePlugin({ publicBaseUrl })

Without it, serve URLs are relative (/_voltro/storage/:id) — correct only same-origin. For public objects a cdnBaseUrl / STORAGE_CDN_URL already yields an absolute CDN URL; VOLTRO_PUBLIC_URL covers the no-CDN + private cases.

To let a cross-origin browser fetch()/canvas the bytes (a plain <img> needs nothing), allow its origin:

STORAGE_ALLOWED_ORIGINS=https://app.example.com   # comma-separated, or '*'

Diagnostics: on boot the plugin logs storage: provider=… presign=… publicUrl=… transforms=…, and warns loudly when provider=memory outside tests (uploads are in-memory and lost on restart — empty/placeholder storage env silently falls back to memory) or when serve URLs would be relative on a non-presigning provider with no public base.

#Diagnostics — voltro storage

Probe the configured backend from the terminal so a broken setup fails LOUD before an upload does:

voltro storage doctor   # provider + a real read/write round-trip + config sanity
voltro storage cors     # print the recommended bucket CORS for presigned uploads

doctor reads your storage env (STORAGE_PROVIDER / S3_* / VOLTRO_PUBLIC_URL / STORAGE_ALLOWED_ORIGINS), prints the resolved provider banner, does an actual put→get→delete round-trip against it, and flags the two silent footguns — provider=memory (data lost on restart) and relative serve URLs (cross-origin breakage). Exit code is non-zero when it finds an issue, so it drops into CI.

cors prints an S3/R2 CORSRules JSON to apply on your bucket — only needed if you upload via a presigned direct-to-bucket PUT (storage.mintUploadUrl); the through-app upload route sets its own CORS from STORAGE_ALLOWED_ORIGINS.

Dev restart-safety. memory (the empty-env fallback) loses everything on a process/pod restart. For a dev environment that survives restarts, use filesystem with a mounted volume (STORAGE_ROOT → a Docker/k8s volume — a ReadWriteOnce PVC is fine for a single dev replica) or the database provider (bytes in your DB, zero extra infra). voltro storage doctor warns when you're on memory so this never surprises you in a deployed dev cluster.

#Who may upload

The self-service upload routes — mintUploadTicket, mintPresignedUpload, beginResumableUpload, beginMultipartUpload, upload — are open on the wire: anyone who can reach the api may call them, and there is deliberately no storage:upload scope, because a scope every user of the app must hold is a rubber stamp rather than a decision.

What bounds them is attribution. Each mint binds the calling subject and tenant into a signed ticket, the finalize step refuses a ticket minted by someone else, and upload / ingestUrl set ownerId to the caller. All of that needs a caller to exist. A request with no session resolves to an anonymous subject whose id is null, and a ticket minted from one carries sub: null, tenant: null — so the object it finalizes has no owner and no tenant, no later owner check, tenant filter or quota can reach it, and the token's subject binding compares null to null and passes for everybody.

So an unidentified caller is refused with a 401. Apps that upload behind a login need nothing: their callers have sessions.

If your app genuinely accepts uploads from visitors who are not signed in — a contact form attachment, an unauthenticated bug report — say so, and pair it with the constraints below:

storagePlugin({
  anonymousUploads: true,
  limits: { maxBytes: 5 * 1024 * 1024, allowedContentTypes: ['image/*'], sniff: true },
  scan: clamavScanner({ host: '127.0.0.1', port: 3310 }),
})

Objects minted that way are ownerless by declaration rather than by accident, and the only thing standing between such a ticket and its object is the signature and the ttl.

#Upload constraints

Reject bad uploads before the bytes ever reach the provider:

storagePlugin({
  limits: {
    maxBytes: 10 * 1024 * 1024,                       // 10 MB cap
    allowedContentTypes: ['image/*', 'application/pdf'], // exact or `type/*`
    sniff: true,                                      // magic-byte check
  },
})

sniff verifies the bytes' magic number matches the declared contentType, so an executable can't slip in as image/png. A violation fails put with a typed StorageRejected — reason is too-large, content-type-not-allowed, or content-mismatch.

#Virus scanning

A scan hook runs on every put BEFORE the bytes are stored. The first-party clamavScanner talks to a running ClamAV daemon over its INSTREAM protocol (no npm dependency):

import { storagePlugin, clamavScanner } from '@voltro/plugin-storage'

storagePlugin({ scan: clamavScanner({ host: '127.0.0.1', port: 3310 }) })

A detection fails put with a typed StorageScanRejected (carrying the threat name) and stores nothing. A scanner outage fails closed — a transient StorageError, so a misconfigured scanner never silently waves files through. Bring your own engine (VirusTotal, Cloudmersive, an ICAP gateway) by implementing StorageScanner:

const myScanner: StorageScanner = async ({ bytes, contentType }) => {
  const clean = await scanSomehow(bytes)
  return clean ? { clean: true } : { clean: false, threat: 'detected' }
}

Scanning applies to every path that runs put — through-app uploads, the resumable finalize, AND prefer: 'presign' (which fetches the bytes back and runs finalizeUpload, so a presigned upload is scanned out of band and fails closed on a hit). Only a raw storage.mintUploadUrl PUT that you finalize yourself bypasses the scan — prefer the hook's presign transport, which does not.

#On-the-fly image transforms

Resize / re-encode images via query params on the serve URL — powered by the optional sharp dependency:

<img src="/_voltro/storage/file_01j…?w=400&h=300&format=webp&q=80" />

Params: w, h (px), format (webp / avif / jpeg / png), q (quality 1–100), fit (cover / contain / …), fp (a focal point x,y as fractions 0–1). Transformed variants are cached.

With w, h and fit=cover (the default), fp decides what the crop keeps. The largest region of the target aspect ratio is cut so the point sits as near its centre as the image edges allow, then scaled. A portrait in a wide banner keeps the face instead of the middle of the shirt. Without fp the crop stays centred. An fp outside 0–1 is ignored.

The fast path stays fast: without transform params a public object 302s straight to the CDN (when cdnBaseUrl is set) or streams with the immutable cache otherwise; with params the app streams + transforms (opt-in cost only when you ask for it). Private objects are access-checked first, then transformed. If sharp isn't installed the original bytes are served unchanged. For CDN-direct public images at scale, a CDN-level image resizer (Cloudflare Images, etc.) avoids the app round-trip entirely.

#Dashboard

Inspect share/revoke requests validate identifiers and supported principal/permission values before writing. Optional expiry must be a positive finite number of days and produce a valid date; malformed requests return HTTP 400 without creating a grant.

Grant and revoke actions prevent duplicate submissions and competing writes while pending. Successful requests show a confirmation; the access list continues to follow the app’s data source, including later changes made by other operators. A failed request retains its input for retry.

The File storage view in local devtools and Cloud pages through references with visibility, size and owner, 50 per page. Search by key, ID, content type, owner or tenant across the complete reference index. Overview statistics report per-tenant byte and reference totals plus public/private reference counts. The framework's metadata stores aggregate the complete reference index; objects in the provider bucket without a registered reference are excluded. Select an object to inspect its policy and grants. Operators with inspect write access can grant access or confirm a revocation; read-only viewers cannot. Custom hosts derive Capabilities.canManageStorage from that authorization.

Multiple storage instances require an explicit selection. Reads and writes use the chosen instance's effective inspect mount and discard the previous instance's object and form state. The default instance exposes /_voltro/inspect/plugins/storage/{refs,stats,grants,share,revoke}.

#Lifecycle: quotas, GC, URL ingest

• Per-tenant quotas: storagePlugin({ quota: { maxBytes, maxCount } }) — the headroom is consumed ATOMICALLY (a compare-and-set over the _voltro_storage_usage counter row) BEFORE bytes are stored, so concurrent uploads — even across replicas — cannot overshoot the cap; a failed write refunds the reservation, a delete releases it. Over-budget fails with a typed StorageRejected ('quota-exceeded'). storage.usage(tenantId) returns { bytes, count } for dashboards / limits.
• Orphan GC: storage.sweepOrphans({ limit, dryRun }) removes dangling refs whose bytes are gone from the provider (a crash between store + insert, or a manual blob delete). Bounded + repeatable; run it from a scheduled task. Only a successful provider HEAD returning absence permits removal. A failed HEAD (including unavailable service or credentials), metadata delete or quota release fails the sweep; it is not reported as successful cleanup. Earlier removals in that sweep are not rolled back. Deletion and quota release are not yet atomic; retrying a sweep cannot recover an already-removed ref's failed quota release.
• Ingest an external URL: storage.ingestUrl(url, opts) (also the storage.ingestUrl rpc) fetches a URL server-side and stores it as an asset — the one-liner for CMS migration ("adopt 21k legacy image URLs"). The server fetches the URL, so every address — the URL and each redirect, which are followed by hand — is checked first: loopback, private and link-local ranges (cloud metadata), localhost and .internal/.local hosts are refused with StorageRejected (blocked-source) before a byte is fetched or stored. Name a deliberate internal source (or a local test server) in storagePlugin({ ingest: { allowHosts: ['assets.internal', '127.0.0.1:8080'] } }). A hostname that only RESOLVES to a private address (DNS rebinding) is not caught here; that needs egress control at the network layer.

URL ingestion enforces storagePlugin({ limits: { maxBytes } }) while reading decoded HTTP chunks, before storing anything. Without an explicit byte limit, URL imports have a 64 MiB cap. Configure a finite, non-negative safe integer for another size; 0 accepts only an empty body. This default applies to URL imports, not every upload method. StorageRejected with reason: 'too-large' closes an oversized response. A missing or understated Content-Length does not bypass the check, and compressed responses are limited by their decoded bytes.

The import buffers at most the configured payload size before the existing scan/checksum/normalization/put pipeline. Buffer growth and normalization can require additional memory: this is bounded collection, not zero-copy streaming to the provider or an exact process-RSS ceiling. Provision memory for the configured cap and concurrent imports. A byte cap is not an SSRF policy or a timeout; keep network allow-lists and bound the owning job/request's duration.

Effect interruption aborts fetch and body reads, including after response headers arrive. Direct server calls additionally accept abortSignal; an already-aborted signal performs no fetch. Cancellation interrupts the entire ingest, stops its put pipeline and runs its finalizers, rather than returning a retryable provider failure. It does not prove that an external provider undid an in-flight write. The RPC uses the same bounded read path but does not serialize a signal. Remote failures expose status/category, not a signed source URL or raw response.

import { Effect } from 'effect'
import { StorageService, storagePlugin } from '@voltro/plugin-storage'

export const plugins = [storagePlugin({ limits: { maxBytes: 32 * 1024 * 1024 } })]

export const importAsset = (url: string, abortSignal: AbortSignal) =>
  Effect.gen(function* () {
    const storage = yield* StorageService
    return yield* storage.ingestUrl(url, { abortSignal, tenantId: 'acme' })
  })

#Schema

_voltro_storage_refs (id, tenantId, ownerId, bucket, key, contentType, size, checksum, visibility, accessPolicy, passwordHash, createdAt, + width, height, duration, placeholder, folder, tags, alt, caption, derivedFrom, kind), _voltro_storage_grants (id, refId, principalType, principalId, permission, createdAt, expiresAt, createdBy), _voltro_storage_usage (id, tenantId UNIQUE, bytes, count, updatedAt — one counter row per tenant, the atomic arbiter for quota enforcement), and _voltro_storage_blobs (id, key, data BYTEA, contentType, size — only written by the database provider) are framework-internal and auto-migrated on every SQL-backed app — no *.entity.ts needed. passwordHash is server-only and never serialized. Behind a put with an operationId sit five more: _voltro_storage_operations (one row per operation: fingerprint, generation, physical key, state, retained delete key), _voltro_storage_object_heads (which generation of a content-addressed object is live), _voltro_storage_put_intents (the ref a reservation will become), _voltro_storage_object_claims (one row per operation and claimant) and _voltro_storage_operation_locks (one row per tenant, the transaction's serialisation point).

#Metadata capacity and upgrades

Object sizes in _voltro_storage_refs.size and _voltro_storage_blobs.size, tenant usage in _voltro_storage_usage.bytes and .count, and the shared-object generation counter _voltro_storage_object_heads.reservations use SQL BIGINT storage. There is no signed 32-bit metadata ceiling at 2,147,483,647 bytes (just below 2 GiB). SQLite and Turso use their existing 64-bit INTEGER representation. The storage API still returns JavaScript numbers: sizes and usage counters must be non-negative safe integers, up to Number.MAX_SAFE_INTEGER (9,007,199,254,740,991). An unsafe value or cumulative reservation is refused instead of rounding the value used for quota accounting. Reservations exceeding that ceiling return StorageRejected with reason: 'quota-exceeded', even without a smaller configured quota. Storage statistics also include BIGINT sizes when metadata uses the in-memory data store; their public byte totals remain safe JavaScript numbers. An invalid stored value or an unsafe statistics total fails with a non-transient StorageError; it is not reported as zero or a rounded total.

Metadata capacity does not change upload limits, provider limits or memory requirements. URL ingestion still defaults to 64 MiB, and its existing scan/checksum/normalization pipeline buffers the payload. Widening a size column does not make a buffered upload of a multi-gigabyte object safe for the process.

When upgrading a SQL-backed app from the older 32-bit storage declarations:

The framework metadata declarations now type these columns as decimal strings, following the public bigint() column contract. Custom code using storageObjectLifecycleTables, raw storage rows or inferred metadata row types must update number assumptions, including the generation's reservations field. The supported StorageRef, ObjectMeta and usage APIs continue to return validated, exact safe numbers. voltro update supplies manual upgrade instructions for direct metadata consumers; it does not replace applying the database schema change.

1. Upgrade the framework packages together so the declared storage schema and runtime agree, then run voltro db plan against the intended database.
2. Review and apply the widening through the normal migration pipeline. Development uses voltro db apply; production uses the explicit deploy step before starting the upgraded serving processes.
3. Run voltro db plan --check after applying; it must report no pending operations before resuming uploads.

Keep the framework's declarations as the source of truth. Manually widening columns while keeping an older runtime leaves a schema mismatch: the old planner sees a reverse change to integer and refuses it as a blocked narrowing. It is not a supported way to keep an old declaration and a different live schema. An older production boot can still pass its unchanged declaration fingerprint without introspecting the live columns; that green boot does not validate a manual widening. An updated runtime must adopt the widened columns through the same reviewed planner path; do not redeclare framework-owned tables in application entities.

Metadata failures retain the safe database error code. Numeric overflow is identified as Storage metadata numeric value out of range, including errors such as PostgreSQL 22003, MySQL/MariaDB ER_WARN_DATA_OUT_OF_RANGE and SQL Server 248/8115. The message does not expose SQL text or failing row contents. For a durable put or delete it also names the unconfirmed phase and tells you to retain the operation identity. An overflow is non-transient: repair the schema or invalid counter value before retrying. Retaining an identity allows recovery of an uncertain operation; it does not repair a capacity mismatch.

The /refs inspect endpoint accepts search (up to 200 characters) and offset. It returns the applied search/offset and nullable nextOffset in page. Search is case-insensitive; paging is ordered by creation time and ID, newest first. The database-backed inventory applies search and pagination in the store query. Objects not registered in the reference index are not included.

#Local dashboard binary upload transport

The local dashboard server provides POST /api/dashboard/proxy/upload for upload preparation and a one-use PUT URL for the file bytes. This transport is separate from the JSON inspect proxy and works when only the dashboard port is forwarded. The shared storage panel provides an upload form when the host grants storage upload access. Select a non-empty file, a storage key, and explicit tenant and owner values (or deliberately select no tenant/no owner). Uploads are private by default; public visibility requires a separate selection. The form prevents duplicate submissions while uploading, keeps inputs after errors, and refreshes the inventory after success.

Preparation accepts the target app's /_voltro/inspect/plugins/:inspectSlug/upload-ticket URL plus explicit tenantId and ownerId (each may be null), key, contentType, positive integer size, and visibility (private or public). The dashboard forwards the app's separate read and write credentials to that endpoint. The returned storage capability stays server-side; the browser receives a random, session-bound URL valid for 30 seconds and one transfer. Send the exact file bytes to it with PUT.

The transfer accepts only the issuing app's storage upload mount, including named instances. Persistent inspect credentials and dashboard cookies are not forwarded with the binary request. Declared and received sizes must match. Cross-site requests and redirects are refused; the transfer times out after two minutes. An interrupted response does not establish whether the object was stored: inspect the file list before preparing another upload.

Storage inventory, statistics and grants expose process-scoped Inspect observations. Replica selection can read them individually or together without treating a valid response as a missing replica. The scope remains conservative because custom metadata stores can be local to a process; an unavailable replica remains visible as an incomplete result.

When several replicas are selected, the Storage panel shows a separate, labelled observation for each replica instead of adding counts or merging file lists from potentially different metadata stores.

The local dashboard and the Cloud host of the shared dashboard carry the selected replica into storage delete, grant, revoke and download operations using the single-target Inspect path. A multi-replica selection cannot dispatch these actions as a batch; use the individual replica panel. Download preparation and the subsequent streamed download retain the same target. Uploads bind admission and byte transfer to the same replica through a signed ticket and the binary relay. Cloud-hosted actions retain the existing tenant, membership and write-role checks before contacting the selected replica.

Inspect-issued upload tickets contain the issuing replicaId in their signed payload. The ordinary upload handler rejects use on a different process with HTTP 409 before storing bytes, even when replicas share a signing secret. This protects process-local metadata; the dashboard upload transport routes the bytes to the issuing replica through the binary relay. Ordinary application upload tickets without this optional binding retain their existing behavior.

#Private links across storage instances

Signed private-file links are bound to both the reference ID and the issuing storage HTTP mount. A link from the default, named, or aliased instance cannot read another instance even when independent reference stores contain the same ID. Keep each generated link on its original mount. After updating, regenerate previously issued private links without this binding through the normal authorized URL-minting flow; they are refused. Provider-native presigned URLs retain their provider authorization.

If an upload response is lost, Check stored files searches for the attempted object key and refreshes the file inventory and statistics without uploading again. The selected file and attribution remain in the form. This read check does not prove that a missing entry was never stored, and it is not an idempotent-upload receipt; inspect the result before explicitly retrying.

#Durable inspect upload identity

The storage inspect POST /upload-ticket endpoint accepts an optional nonblank operationId of at most 200 characters. Renew tickets with the same ID and identical file bytes and metadata to reuse the stored reference instead of charging quota and creating another reference. Identity is scoped to the storage plugin instance. Changed content under the same ID is rejected. This requires a bound data store and the default reference/grant stores; otherwise admission returns 409 before bytes are sent.

The shared dashboard upload form saves the attempt identity and attribution in browser local storage before sending bytes. After an uncertain response or page reload, reselect the original file and submit to recover the result with the same identity. Attribution remains locked until success or explicit confirmation of Start another upload. File name, size and content type are checked locally; the storage service also checks the actual content. Browser storage failures prevent an untracked upload.

After confirmed success, the form removes the saved attempt. Starting another upload explicitly discards recovery information; the previous file may already exist. Check stored files remains a read-only inventory check, not a receipt. Clearing browser storage also removes recovery information. Tabs on the same dashboard origin coordinate each destination through Web Locks. A competing upload or recovery-discard action is refused immediately while a transfer is active. A stale tab cannot silently restart an attempt completed or replaced by another tab. This requires Web Locks in a secure browser context (HTTPS or localhost); without it, the form refuses to send. Different browser profiles or dashboard origins do not share this coordination.

Custom hosts must provide a stable StoragePage.uploadScope derived from app identity, endpoint, storage instance and replica selection. Change it when the destination changes; never include credentials or use a shared constant or random value. Custom upload adapters must forward StorageUploadInput.operationId unchanged. The local dashboard proxy and shared Cloud host transport already preserve and validate it.

Damaged saved recovery information is shown explicitly. After checking the inventory, confirm Start another upload to discard it. This does not send a file. If the saved record changed in the meantime or browser storage refuses removal, it is preserved and an error is shown.