The adding jitter strategy was one of the most commented on techniques from 7 Years Of YouTube Scalability Lessons In 30 Minutes on HackerNews . Probably because it's one of the emergent phenomena that you really can't predict and is shocking when you see it in real life. Here's the technique:
Add Entropy Back into Your System
If your system doesn't jitter then you get thundering herds . Distributed applications are really weather …
…GCGraceSeconds , you should treat it as a failed node and replace it as described in Cassandra Operations . The default setting is very conservative, at 10 days; you can reduce that once you have Anti Entropy configured to your satisfaction. And of course if you are only running a single node, you can reduce it to zero, and tombstones will be GC'd at the first compaction .