Design a Unique ID Generator in Distributed Systems

The 4 Golden Rules of a Good ID

Before picking a solution, let's define what our ID generator must deliver:

• 1. Guaranteed Unique: No two items anywhere in the world can ever be assigned the same ID.
• 2. Small & Numeric (64-Bit Integer): Numbers take up very little space in computer memory and on hard drives. A standard 64-bit number is compact, fits easily in database columns, and makes search queries much faster than long text strings.
• 3. Sortable by Time (Roughly Increasing): An ID created at 2:05 PM should be a bigger number than an ID created at 2:00 PM. This lets social media feeds, chat apps, and user dashboards display recent items in chronological order automatically, without needing to run slow sorting queries.
• 4. Blazing Fast & Always Available: Each server should be able to create an ID in its own local memory in nanoseconds. If the network hiccups or a server crashes, the rest of the system must keep running without pausing.

Why Random UUIDs Slow Down Databases

When developers first face this problem, their first thought is often: “Why not just generate random UUIDs?”

A random UUID (like f47ac10b-58cc-4372-a567-0e02b2c3d479) is a 128-bit random string. Because it is so long, the chance of two servers ever creating the same UUID by accident is virtually zero. You don't need any central coordination.

However, using random UUIDs as your main database ID comes with major hidden drawbacks:

• Disorganized Storage (Page Splitting): Databases store records like books neatly arranged on a shelf in order. When IDs are sequential (1, 2, 3...), the database simply adds each new book to the end of the shelf. But with completely random UUIDs, each new item needs to go into a random spot in the middle. When a shelf fills up, the database has to stop, push half the books to a new shelf, and reorganize everything. This causes huge disk slowdowns.
• Wasted Memory Cache: Because new items land randomly across thousands of different disk locations, the database constantly has to fetch new pages from slow hard drives into RAM, kicking out useful cached data.
• Twice the Storage Size: A 128-bit UUID string takes up significantly more storage space than a neat 64-bit number, bloating every table and index in your system.

We need an ID that is generated independently by each machine, but still stays short, numeric, and sorted by time.

1. Taking Turns (Multi-Master Auto-Increment)

One intuitive idea is to have your database servers take turns by counting in steps:

• Server 1: Generates odd numbers (1, 3, 5, 7, 9...)
• Server 2: Generates even numbers (2, 4, 6, 8, 10...)

If you have k servers, each server simply adds k every time it creates a new ID. They never give out the same number, and they don't have to check in with each other.

Where Taking Turns Breaks: The Scaling Problem

This sounds simple, but it quickly falls apart in practice:

• Pros: Easy to set up; uses built-in database counters; no custom coordination software required.
• Where It Breaks (Cons):

  • Hard to Add More Servers: If your traffic doubles and you add a 3rd server, you now need all servers to count by steps of 3 instead of 2. Changing this rule on live databases creates immediate overlaps with old numbers or leaves huge, permanent gaps.

  • Time Order Gets Lost: If Server 1 receives 5,000 requests while Server 2 only receives 10 requests, Server 1's counter shoots way ahead (e.g. ID 10,001) while Server 2 is still giving out small numbers (e.g. ID 22). A newer post might end up with a much smaller number than an older post!

2. The Central Ticket Booth (Flickr Pattern)

To keep IDs strictly sequential, Flickr built a famous design called the Ticket Server .

Instead of letting individual app servers generate IDs, they set up one dedicated database server whose only job in life is to act as a ticket booth. Whenever an application server needs an ID, it sends a quick message to the ticket booth: “Give me the next number!”

Where the Ticket Booth Breaks: Bottlenecks & Outages

• Pros: Generates perfectly ordered, gap-free 64-bit numbers; very simple to understand.
• Where It Breaks (Cons):

  • Single Point of Failure (SPOF): If that one ticket booth server loses power, crashes, or has a network problem, your entire application stops working because no new posts, orders, or users can be saved.

  • Network Delay on Every Write: Every single action must wait for a round-trip message across the network just to get an ID before doing real work, adding noticeable lag to every user action.

  • Traffic Ceiling: One central computer can only handle so many messages per second before it runs out of CPU and memory.

The 64-Bit Recipe: Assembling the Puzzle

Think of a Snowflake ID like stamping a unique code made of 5 pieces of information:

• 1. Sign Bit (1 Bit): Always set to 0. This ensures the number is always positive in programming languages.
• 2. Time (41 Bits): The current time measured in milliseconds since a custom starting date (epoch). Because the timestamp sits at the front of the number, all Snowflake IDs are naturally sorted by time!
• 3. Datacenter ID (5 Bits): Identifies which building or cloud region the server is in (supports up to 32 datacenters).
• 4. Machine ID (5 Bits): Identifies the specific computer inside that datacenter (supports up to 32 machines per datacenter, for a total of 1,024 servers).
• 5. Local Counter (12 Bits): A simple counter that ticks up for every ID created on that exact machine during the same millisecond (allows up to 4,096 IDs per millisecond). It resets back to 0 when the next millisecond starts.

The Numbers: 69 Years and Millions of IDs Every Second

Here is why this bit structure is so powerful:

• 69 Years of Lifespan: 41 bits of millisecond time gives you roughly 69 years of unique timestamps. If we set our custom start date to January 1, 2026, the system will run happily until the year 2095 before running out of time bits!
• 4 Million IDs Per Second Per Machine: Because each server has a 12-bit counter (4,096 numbers every millisecond), a single machine can generate over 4 million unique IDs every second.
• 4 Billion IDs Per Second Globally: Across all 1,024 servers in your cluster, the system can generate over 4.19 billion IDs per second, with zero network delay and zero cross-server communication.

1. What If the Clock Jumps Backward?

Snowflake depends completely on the computer's internal clock. But physical clocks on computer chips drift slightly over time.

To stay accurate, servers regularly sync their clocks with internet time servers (NTP). If NTP notices that a server is 50 milliseconds ahead of true time, it might suddenly step the server clock backward .

If a server has already generated IDs up to timestamp 200ms, and the clock is suddenly rewound back to 150ms, the server will start creating IDs using timestamps it already used. This creates duplicate IDs and can corrupt your database!

The Solution: The generator always remembers the last timestamp it used. If it sees that the current time is slightly behind the last time, it pauses and waits a few milliseconds for the physical clock to catch up. If the clock jumped backward by a large amount, it raises an alarm and refuses to generate IDs until an engineer investigates.

2. What If Traffic Spikes in a Single Millisecond?

Each server can generate up to 4,096 IDs in one millisecond.

If a massive traffic surge hits and a single server receives 5,000 requests in just 0.3 milliseconds, its counter will reach 4,095 and run out of numbers for that millisecond.

The Solution: If the counter maxes out, the server simply pauses for a fraction of a millisecond until the clock ticks to the next millisecond, resets the counter to 0, and continues. The user only waits a tiny fraction of a millisecond.

3. What If Two Servers Claim the Same Name?

Snowflake relies on every machine having a unique Machine ID (from 0 to 31).

In modern cloud setups (like Kubernetes or Docker), servers restart automatically. If two new server instances accidentally boot up with the same Machine ID (e.g. both claim Machine 5), they will create identical IDs.

The Solution: Do not type machine IDs by hand. Instead, use an automatic cluster coordinator (like ZooKeeper or etcd). When a server starts up, it asks the coordinator for a temporary lease on an available Machine ID. If the server crashes, the lease automatically expires and is freed for someone else.

4. The 69-Year Expiration Clock

Because timestamp space is limited to 41 bits, the clock runs out of room after approximately 69 years.

If you start counting from the standard computer epoch (1970), the numbers will overflow in 2039.

The Solution: Set your starting point (epoch) to the day your project launches (e.g. 2026). This gives your application 69 years of worry-free runway.

UUIDv7: The Modern 128-Bit Standard

Approved officially in 2024 (RFC 9562), UUIDv7 solves the old UUID problem by putting the timestamp right at the front of a 128-bit UUID.

Because time is at the front, UUIDv7 fixes database clustering and eliminates slow disk splits, while requiring zero machine coordination. It is rapidly becoming the default ID standard in modern databases like PostgreSQL.

ULID: Human-Readable Sortable Strings

ULID packs a timestamp and randomness into a neat 26-character text string (such as 01ARZ3NDEKTSV4RRFFQ69G5FAV).

ULIDs are case-insensitive, avoid letters that look alike (like 0 and O, or 1 and l), and are ideal when IDs need to appear directly in website URLs or application logs.

Which ID Strategy Should You Choose?

Here is a quick breakdown to help you pick the right tool for the job: