Dropbox’s Exodus From the Amazon Cloud Empire

Cade Metz published a Dropbox profile for Wired, detailing how the company migrated to their own storage infrastructure:

Cowling and crew started work on the Magic Pocket software in the summer of 2013 and spent about six months building the initial code. But this was a comparatively small step. Once the system was built, they had to make sure it worked. They had to get it onto thousands of machines inside multiple data centers. They had to tailor the software to their new hardware. And, yes, they had to get all that data off of Amazon.

The whole process took two years. A project like this, needless to say, is a technical challenge. But it’s also a logistical challenge. Moving that much data across the Internet is one thing. Moving that many machines into data centers is another. And they had to do both, as Dropbox continued to serve hundreds of millions of people. “It’s like a moving car,” says Dan Williams, a former Facebook network engineer who oversaw much of the physical expansion, “and you want to be able to change a tire while still driving.” In other words, while making all these changes, Dropbox couldn’t very well shut itself down. It couldn’t tell the hundreds of millions of users who relied on Dropbox that their files were temporarily unavailable. Ironically, one of the best measures of success for this massive undertaking would be that users wouldn’t notice it had happened at all.

People who are really serious about cloud storage should make their own hardware and software, I guess.