Probabilistic and distributed systems · 49 / 53
Consistent hashing and placement balance
Understand → trace → implement → justify
2 min reading · allow extra time for practiceModulo routing h(key)%N is simple, but changing N relocates many keys. Consistent hashing places tokens and keys on a ring and selects the next clockwise token. Adding a node moves only affected intervals. Virtual nodes improve placement balance and weighting, but do not automatically solve hot-key load.
Rendezvous hashing scores each key-node pair and selects the maximum. A new node moves only the keys it wins. Every participant needs the same stable seed and hash; process-randomised runtime hashes are unsuitable for routing.
Placement does not provide replication consistency. Replication factors, failure domains, membership, migration, tombstones, read repair and quorums need separate design. Balanced key counts do not ensure balanced request load. Not every load balancer uses consistent hashing.
A worked trace
When D joins A,B,C, changed rendezvous assignments move to D, rather than unnecessarily between existing nodes.
Cost and assumptions
Naive rendezvous costs O(N) scores per query; a ring binary search costs O(log T) for T tokens.
Executable example
Run with Python 3. Assertions check a small example; they do not constitute a general correctness proof. Some advanced examples are explicitly small models or reference oracles, not full production implementations.
import hashlib
def route(key,nodes):
return max(nodes,key=lambda node:hashlib.sha256((key+":"+node).encode()).digest())
for i in range(100):
key=str(i); old=route(key,["A","B","C"]); new=route(key,["A","B","C","D"])
assert old==new or new=="D"TEST YOUR UNDERSTANDING
Your turn
What if servers use different seeds for the same key?
Reveal the reasoning
Routing and ownership disagree. Seeds and membership versions belong to the protocol.
Take it further
- Write down the input contract and the invariant before changing the example.
- Add empty, boundary, duplicate and adversarial cases where they are relevant. Compare with a simple oracle.
- Derive the cost under the stated model; measure several input sizes and explain discrepancies.
Primary reading & lecture notes
Original explanations in this course; these references provide proofs and further reading. Source links may cover the broader module.
MIT 6.046J · Design and Analysis ↗