Re:How to use Amazon’s S3 web service for Scaling Image Hosting
by swindsor on Wednesday July 14, 2010 @07:21PM
(#1516299)
1. Good point - I didn't bring this up in the article. When we store the images locally, we create divide end of the unique ID of the image into two levels of subdirectories. This gives us "random enough" distribution in subdirectories, and an easy way to look up the files on disk. This logic exists in our rails app, but we didn't want to expose this pathing out to end users. This allows us to change it later if we need to create further subdirectories.
2. Good tip - we try to use our judgement here balancing good design/good performance. We tend to opt for a pretty website first, then go back and make it faster with optimization.
Re:How to use Amazon’s S3 web service for Scaling Image Hosting
by zepolen on Wednesday July 14, 2010 @08:32PM
(#1516453)
Regarding "This logic exists in our rails app, but we didn't want to expose this pathing out to end users". I'm not sure if you understood, but I'm saying exactly this. nginx will take a /whatever request and convert it to /w/wh/wha/whatever on disk transparently - you can (and I have needed to) change this whenever you like with no front facing changes.
Re:How to use Amazon’s S3 web service for Scaling Image Hosting
by swindsor on Wednesday July 14, 2010 @08:41PM
(#1516464)
Oh, that's cool - I didn't know you could use matches from a location in an alias. Is that a new feature in nginx?
That way deploying more photo servers for speed just requires nginx and a few lines of config.
A couple notes:
1. Make sure you partition your data on your local disk properly so that there are never more than ~5000 files per directory, you can use nginx rewrite rules to do this for you (and make nginx save the actual url to nested folders).
2. If possible, make thumbnails smaller than 4kb.