You might also want to try thttpd: http://www.acme.com/software/thttpd/ I heard all the porn sites use it to host their images.

With only the http server for the images running it should work quite well to rely on the filesystem cache.

http://www.danga.com/mogilefs/

with an nginx or lighttpd frontend.

The documentation is a little sparse though the mailing list and irc are useful for help.

Also it would somewhat depend on your traffic profile. If you've got thousands of random reads (e.g. thumbnails with dozens per page, ssds might be better; specifically the Intel X-25).

Otherwise MogileFS with an array of fast drives would probably do the job. Mogile can also do load balancing based on drive I/O because each image is replicated across different files/machines.

EDIT: Make sure you've got browser caching set up correctly (though admittedly that's a tiny bit harder with Mogile)

If you can use a expiry time that's far in the future. Also don't send a Last-Modified nor an ETag Header. Only send Expires.

Facebook - Needle in a Haystack: Efficient Storage of Billions of Photos

http://www.flowgram.com/p/2qi3k8eicrfgkv/

Super duper secure and very, very quick, especially for smaller files.

I'd try and run some tests against it, thttpd and your existing apache setup.

My own 200mbps connection costs me the equivalent of 300$ a month.

Thanks, but I was looking for technology to install on my own server. Not moving to Amazon S3