Link Details

About a year ago we decided to scale our database horizontally - that is, partition it. We had many millions of users in the database, and we were contemplating allowing a lot more user-generated content on our site, as well as collecting much more data on what our users were doing. We had been burned many times by the vertical-scaling strategy ("buy a bigger box") - it's harder and harder to get the money for the next bigger box, you can only get one or two big boxes at a time, everything ends up on that box ("it's the only box powerful enough"), and when it crashes the entire world goes down. So we decided to partition horizontally with commodity hardware.