Deep Dive
system design interviewalgorithmsconcurrencydatabases

Design a Distributed ID Generator

Auto-increment doesn't survive sharding, and random UUIDs wreck your database's index. How do you mint billions of unique, roughly time-sortable 64-bit IDs across a fleet with no coordinator? This is the Snowflake design — bit for bit.

·16 min read
Medium
Asked atTwitterInstagramDiscordAmazonUber

The moment you shard a database, AUTO_INCREMENT stops working — two shards would hand out the same id. This one constraint spawned a whole family of designs, and the reference answer, Twitter’s Snowflake, is elegant enough to reconstruct from first principles on a whiteboard.

Four requirements, and why the obvious answers fail

Database AUTO_INCREMENT
  • One counter → single point of failure
  • A write bottleneck; breaks under sharding
  • Sequential → leaks your total row count
  • A network hop on every insert
Random UUIDv4 (128-bit)
  • No coordination — but not sortable
  • Random → terrible B-tree index locality
  • Random inserts cause page splits & write amp
  • 128 bits — double the storage of a 64-bit id

The winning design threads all four needs: unique across the fleet, roughly time-sortable (so it indexes well and you can page by recency), compact (64 bits, fits a BIGINT), and coordination-free (each node mints locally, no round-trip).

The idea: stop counting, start composing

Snowflake’s insight is that you don’t need a shared counter if you build the ID out of parts that are already unique when combined:

01 bit · signtimestamp41 bits · ~69 yrsmachine id10 bits · 1,024sequence12 bits · 4,096/ms
A 64-bit Snowflake ID. The leading bit is always 0 (keeping the signed integer positive). 41 bits of millisecond timestamp make it time-sortable and last ~69 years; 10 bits name up to 1,024 machines; 12 bits count up to 4,096 IDs within a single millisecond on one machine.

Multiply it out: 1,024 machines × 4,096 IDs per millisecond = ~4.2 million IDs per millisecond, all globally unique, all sortable by time, all minted without a single machine talking to another. That’s the whole trick.

The members-only build takes it apart bit by bit: an interactive 64-bit inspector where you mint IDs and watch the sequence bits climb within a millisecond — then rewind the clock to trigger the failure that keeps Snowflake engineers up at night. Plus the clock-skew problem and how NTP breaks monotonicity, UUIDv7 as the modern coordination-free alternative, Instagram’s Postgres variant, the full multi-language generator, and the interview corner.

Members only

Keep reading with Premium

You've reached the members-only part of this deep-dive — the full implementation, the interactive ring simulator, and the step-by-step walkthrough. Unlock it with a membership.

Go PremiumAlready a member? Sign in