In our previous look at building scalable systems in System Design: The Architecture That Scales , we saw that a single computer running your database will eventually hit a limit. No matter how powerful or expensive that computer is, its hard drive will run out of space, its memory (RAM) will fill up, and it can only handle so many read and write requests per second.
When your application grows to millions of users and terabytes of data, you have no choice but to split your data across multiple separate computers. In system design, each separate database computer is called a database shard (or server node).
Splitting your data across several computers sounds simple, but it quickly brings up a tricky question: When a user logs in and wants to find their profile (like “user_1001”), how does the application know which exact database computer holds that data, without having to ask a slow central manager every single time?
The Simple Division (Modulo) Trap
The most obvious way to solve this is by using basic remainder math, known as modulo division .
You turn the user's ID (like "user_1001") into a number using a standard hash function. Then, you divide that number by the total number of database servers (N). The remainder tells you which server owns the data:
server_number = hash(user_id) % total_servers
Think of it like dealing a deck of cards into 3 piles. If you have 3 database servers (N = 3), and hash("user_1001") turns into the number 2491:
2491 % 3 = Server 1 (since 2491 divided by 3 leaves a remainder of 1)
Any server in your application can calculate 2491 % 3 in a fraction of a millisecond and talk directly to Server 1. It is fast, easy to code, and spreads your users evenly across all 3 servers.
Everything works smoothly- until your servers run out of space .
When your 3 servers get full, you buy a 4th server to add more storage room (N = 3 → 4).
Because the formula now divides by 4 instead of 3, the remainder changes for almost every single user in your database .
User "user_1001" was stored safely on Server 1. But with 4 servers:
2491 % 4 = Server 3
Now, the application looks for user 1001 on Server 3, but the data is physically sitting on Server 1! The system cannot find the user's data anymore.
The Big Problem: Moving Huge Piles of Data
When you change the number of servers from N to N + 1, a staggering N / (N + 1) of all your records will suddenly point to the wrong server.
If you grow from 9 servers to 10 servers, 90% of your entire database has to be physically copied across your network to new machines .
If your database holds 50 Terabytes of data, adding just one machine means moving 45 Terabytes around your data center!
- Network Traffic Jam: Internal cables and switches get overwhelmed moving huge database tables, slowing down real user requests.
- Overworked Drives: Hard drives and SSDs get choked reading and writing gigabytes of migrating data every second, making simple queries take seconds instead of milliseconds.
- Temporary Freezes: To avoid data conflicts while files are being copied, parts of the database have to be locked, causing errors and downtime for your users.
Real Story: Google AdWords Took 2 Years to Move Data
This isn't just a theoretical problem-it happened to one of the biggest systems in the world.
Back in 2005 and 2006 , Google was running a massive database fleet of 90+ MySQL servers to power Google AdWords (the system that makes most of their revenue).
As more businesses signed up for ads, individual servers ran out of room. To rebalance all that data across new servers, Google had to reorganize their entire database cluster.
Because there was so much live customer data and money moving through the system, the migration took 2 full years to complete .
The pain of that two-year migration pushed Google engineers to rethink how data is distributed. It directly inspired them to build modern databases like Google Spanner -systems designed from day one to balance data automatically so engineers never have to go through a painful manual migration again.
The Clever Solution: The Hash Ring
In 1997, researchers at MIT introduced a brilliant solution called Consistent Hashing .
Their goal was simple: when you add or remove a server, only move the small slice of data that belongs on the new machine , leaving the rest of the data sitting undisturbed right where it already is.
Putting Numbers in a Circle
Instead of dividing by the number of servers, consistent hashing arranges both servers and data records on a giant circular ring of numbers.
Imagine a big circle, like the face of a clock, with numbers starting at 0 and going all the way around to 4,294,967,295 (which is 2³² - 1). When you reach the highest number, it simply wraps back around to 0.
The Clockwise Rule
Here is how data is assigned to servers using this circle:
- 1. Place Servers on the Circle: Run each server's name (like
"server-1") through a hash function to get a number on the ring. Place the server at that exact spot. - 2. Place Data on the Circle: Run the data ID (like
"user_1001") through the same hash function to get its position on the ring. - 3. Walk Clockwise: To find which server stores that user's data, start at the data's spot on the circle and walk clockwise . The very first server you bump into is the server that holds that data!
Each server becomes responsible for the slice of the circle between the previous server and itself.
Adding and Removing Servers Easily
The real magic of the circle becomes clear when you need to add or remove servers.
Adding a Server: Only One Neighbor Shares Data
Imagine we have 4 servers on our circle: Node A, Node B, Node C, and Node D. We need more room, so we add Node E.
Node E gets placed at a spot between Node A and Node B. Look at what happens to the data:
- Nodes C and D: Not a single piece of data moves! Every record on their parts of the circle still runs into them clockwise.
- Node A: Zero data movement.
- Node B & Node E: Node B used to manage the whole slice from Node A to Node B. Now, Node E takes over the slice between Node A and Node E. Only the records in that one slice need to move from Node B to Node E.
Think of it like someone pulling up a chair at a round dinner table. They only share food with the person next to them-nobody else at the table has to get up and switch seats!
Removing a Server: A Smooth Handover
If Node C breaks down or is turned off for maintenance, its slice of the circle simply rolls over to the next server clockwise (Node D).
Only the records that were on Node C need to be handled. Every other server keeps doing its job with zero interruptions.
The Math: Why Only a Small Slice Moves
When you add a new machine to an N-server system:
Fraction of data moved = 1 / (N + 1)
Let's compare the two approaches when growing from 9 servers to 10 servers with 50 Terabytes of total data:
- Simple Modulo Division: Moves 45 Terabytes (90%) over the network, causing massive slowdowns and freezes.
- Consistent Hashing: Moves only 5 Terabytes (10%) , leaving the other 45 Terabytes completely untouched on their original drives.
The Uneven Workload Problem
The basic circle sounds great on paper. But if you try to use it with just a few servers, you run into an issue: the work is rarely split evenly .
Why Random Spots Overload Some Servers
When you place only 3 or 4 servers on the circle randomly, they almost never land evenly spaced apart.
Just by random chance, two servers might land right next to each other, while another server ends up responsible for a huge 60% chunk of the entire circle.
That one unlucky server will get 60% of all user requests and files. Its disk will fill up, its CPU will overheat, and if it crashes, all of its data will suddenly dump onto its neighbor, potentially crashing the whole system like falling dominos.
Handling Big and Small Servers
In the real world, your servers might not all be identical. You might have older servers with 2 Terabyte hard drives and newer, stronger servers with 8 Terabyte fast SSDs.
If every server only gets one spot on the circle, the big 8 TB server gets the exact same amount of data as the small 2 TB server, wasting valuable capacity.
The Virtual Spots Solution (Vnodes)
To fix uneven balance and handle different server sizes, modern systems use Virtual Nodes (often called vnodes ).
Instead of giving each physical server just one spot on the circle, you give each server 100 to 256 virtual spots spread all over the ring.
Giving Each Server Multiple Spots on the Circle
To place Server A on the circle multiple times, you simply hash its name with different numbers added to it:
vnode_0 = hash("server-A#0") vnode_1 = hash("server-A#1") vnode_2 = hash("server-A#2") ... vnode_199 = hash("server-A#199")
Now Server A, Server B, and Server C each have hundreds of small slices scattered throughout the circle, thoroughly mixed together.
Achieving Fair Balance Across Servers
Because the slices are small and spread out, random clumps naturally average out:
- 1 Spot per Server: Very uneven. One server might get 60% of the data while another gets only 10%.
- 150–256 Spots per Server: The difference drops to less than 5% . Every physical machine gets an almost perfectly equal share of the work.
Giving More Work to Stronger Machines
Virtual spots also make it effortless to mix different server sizes:
- Give a smaller 2 TB server 100 virtual spots on the ring.
- Give a powerful 8 TB server 400 virtual spots on the ring.
The 8 TB server automatically gets 4 times more data without needing any complicated custom routing rules!
Keeping Copies Safe (Replication)
In production, computers fail and servers need to reboot. If you only save one copy of each piece of data, losing a server means losing user information.
Consistent hashing works alongside replication (saving multiple copies) to keep data safe even when hardware breaks.
Saving Copies on Next Clockwise Servers
If you want to keep 3 copies of every record ( Replication Factor = 3 ):
- First Copy: Stored on the first server you hit walking clockwise on the ring.
- Backup Copies: Continue walking clockwise and store the second and third copies on the next 2 physically separate machines .
Smart database systems make sure these 3 servers are in different server racks or even different buildings (Availability Zones). That way, even if a power outage shuts down an entire building, your data is still safe and available.
Conclusion
At first glance, partitioning data across a cluster feels like a simple math problem: take a hash, compute the remainder, and route to the corresponding server.
Then the cluster grows.
A single server addition turns into a cluster-wide data migration storm. Hard drives saturate, networks lock up, and nodes drop offline under the strain of resharding.
Consistent hashing changes the fundamental geometry of distributed state.
By projecting both keys and storage nodes onto a continuous circular token ring, cluster resizing stops being a destructive global shuffle:
Modulo Sharding: Adding 1 node moves ~N / (N + 1) data (90%+) Consistent Hashing: Adding 1 node moves only 1 / (N + 1) data (10%)
When we introduce virtual nodes , we eliminate uneven token variance and smoothly accommodate mixed hardware. When we add topology-aware replication , the ring gains fault tolerance against rack failures.
The real lesson of consistent hashing is about designing with change in mind.
In distributed systems, machines will join, crash, and restart continuously. The architecture shouldn't fight that reality-it should be mathematically built for it.
Design the ring. Localize the disruption.