My initial thought was: wow, this looks like a great validation of the idea behind Joyent Manta[1] storage+process. I wonder if it would've been a good fit for 500px, and how it would compare in terms of price/performance? Granted, they now have a working system on s3, so maybe for the next up-and-comming competitor? ;-)

Great write-up, btw. And thanks for the heads-up about vips/nip2 -- I wasn't aware of those.

[1] https://www.joyent.com/object-storage