Snowflake — Twitter 64-bit

Snowflake is the framework's high-throughput ID scheme — a 64-bit integer composed of a millisecond timestamp, a machine identifier, and an in-millisecond sequence counter. Use when you're inserting at a rate where TypeID's 30-character string is a meaningful storage cost across hundreds of millions of rows.

import { id } from '@voltro/database'

export const events = table('events', {
  id:        id({ scheme: 'snowflake' }),    // → 1879382109193580544
  userId:    reference(() => users),
  eventType: text(),
  payload:   json(),
  occurredAt: timestamp(),
})

Each process needs a unique SNOWFLAKE_MACHINE_ID:

SNOWFLAKE_MACHINE_ID=42 voltro start

#Spec

Twitter's original Snowflake layout, slightly adjusted for JavaScript:

+--------+------+------+------+
| 1 bit  | 41   | 10   | 12   |
| sign   | time | mach | seq  |
+--------+------+------+------+

• Sign bit: always 0 (positive integers — JavaScript Number can represent up to 2^53 safely, the framework wire-formats as string).
• Timestamp (41 bits): milliseconds since the framework's epoch (2024-01-01T00:00:00.000Z). 41 bits ≈ 69.7 years from epoch → 2093-08-14.
• Machine ID (10 bits): 0–1023. Set per process via SNOWFLAKE_MACHINE_ID env. Unique across all processes that share a database.
• Sequence (12 bits): 0–4095. Resets every millisecond. Allows up to 4096 IDs per machine per millisecond — 4M IDs/sec/machine.

The framework implements this in ~100 LOC at voltro/packages/database/src/snowflake.ts. No external dependency.

#When to pick Snowflake

One constraint: you need MORE than the trade-offs.

• Throughput: hundreds of thousands to millions of inserts per second per process. TypeID's typeid-js generator does ~1M ops/sec on a modern Node, so most workloads don't need Snowflake's tighter inner loop.
• Storage: 8 bytes per ID, vs TypeID's ~30. At 10 billion rows with three FK references per row, that's ~720 GB saved.
• Distributed insert without coordination: with SNOWFLAKE_MACHINE_ID correctly set per process, no two processes ever produce the same ID. No DB round-trip, no central counter, no UUID collision improbability arguments.

If none of these apply, TypeID is the better default.

#Configuration

SNOWFLAKE_MACHINE_ID=0     # process 0
SNOWFLAKE_MACHINE_ID=1     # process 1
SNOWFLAKE_MACHINE_ID=2     # process 2
# ...up to 1023

The framework reads this at boot. If unset on a process running tables that use the Snowflake scheme, boot fails with a clear message:

ERROR: SNOWFLAKE_MACHINE_ID env var must be set to an integer 0–1023.
Reason: a table with id({ scheme: 'snowflake' }) is in the schema +
        no machine id is configured — concurrent processes would
        produce duplicate Snowflakes (silent ID collision).

If you only use TypeID + ULID, you don't need to set this even on tables that interact with Snowflake-using tables.

#Storage

Snowflake values are JavaScript bigint at the runtime layer; the framework wire-formats them as strings on the wire (avoiding the Number.MAX_SAFE_INTEGER = 2^53 precision loss that bites JSON-based APIs).

On the database side:

• BIGINT on every SQL dialect.
• 8 bytes on disk per column.

The PK index is a B-tree. Inserts cluster at the high end of the tree (timestamp-leading) — same B-tree-friendly pattern as TypeID/ULID, no random insertion penalty.

#What you lose vs TypeID

• Prefix tag — 1879382109193580544 carries no table affinity. Brand types still flow at compile time, but logs / error messages can't disambiguate at a glance.
• URL friendliness in the same sense — 19-digit numbers are URL-safe (they're just digits) but they look like database internal IDs. The framework's brand keeps them safe; if you're optimizing for "looks intentional," TypeID wins.
• JavaScript number safety. The largest possible Snowflake ((2^63) - 1) is 9.2 × 10^18, far exceeding Number.MAX_SAFE_INTEGER (9 × 10^15). The framework uses BigInt at the type level and wire-formats as string, but any code path that goes through JSON.parse(string) → Number silently loses precision.
• 70-year epoch ceiling. The framework's Snowflake epoch is 2024-01-01. The 41-bit timestamp gives ~69.7 years → 2093-08-14. Systems that need to outlive that timeline need a new epoch.

#What you DON'T lose

• Sortable by creation time: timestamp is the high-order bits. ORDER BY id works.
• Branded at the TypeScript level: same schema-registry brand as TypeID.
• Cursor pagination via paginateById: BigInt comparison works for cursor windows.

#Caveats

• Machine ID collisions corrupt the ID space silently. Two processes with SNOWFLAKE_MACHINE_ID=42 produce overlapping IDs. The framework can't detect this — it's deployment configuration. Coordinate IDs via your orchestrator's metadata (k8s pod ordinal, Nomad index, ECS task instance).
• Clock skew matters. The framework uses Date.now() for the timestamp. If a process's clock is set backwards (NTP failover, manual adjustment), it can produce IDs that go backwards in the body for a window. The framework defensively waits if the new timestamp is less than the last one used; this manifests as a tiny pause on insert.
• Snowflake is 8-byte. Don't store it as a string in your application code — the framework wire-formats as string but in-process it's BigInt. JSON.stringify(BigInt(1879382109193580544)) throws — handle the BigInt → string conversion explicitly at API boundaries.
• Cross-process clock drift produces non-monotonic IDs across processes. Two writes happening "at the same time" from machines with skewed clocks produce IDs that sort differently than real-world time. Don't rely on cross-process Snowflake order for audit purposes; use a centralized timestamp source if order across processes matters.

#Where it lives

• voltro/packages/database/src/snowflake.ts — Twitter-spec implementation, ~100 LOC; exports generateSnowflake()
• voltro/packages/database/src/idGenerator.ts — generateId(scheme, tableName) calls generateSnowflake() for the snowflake scheme
• voltro/packages/database/src/columns.ts — id({ scheme: 'snowflake' }) registration
• voltro/packages/runtime/src/storeMiddleware.ts — boot-time machine-ID check + per-insert auto-injection