News for hackers. Stuff that matters.
Re:How to Use Varnish Cache as Secured AWS S3 Gateway by andrewguenther on Friday February 19, 2016 @01:20PM (#11135333)
I don't really buy this article. They claim you get "caching, more efficient bandwidth utilization, centralized access with logging and security." S3 gives you caching already. I fail to see how your bandwidth utilization improves, if anything you've introduced another hop (depending where you're hosting Varnish). Centralized access? I guess, if you're already using Varnish for other things, but if you're at the point that you need Varnish, you probably are aggregating multiple log sources already. Security? You're just adding another possible exploitation layer. And on top of that you've just added another box to maintain.

Not to rag on Varnish, I love it, I just don't really see this use case. Personally, I would never route plain S3 data through Varnish.

Re:How to Use Varnish Cache as Secured AWS S3 Gateway by true_religion on Friday February 19, 2016 @02:19PM (#11135849)
One use case: reducing ongoing bandwidth costs.

S3 bandwidth is pretty expensive. Even Akamai will give you less expensive bandwidth and their target market is enterprise media companies (e.g. ESPN) with deep pockets.

For someone with bandwidth costs issues who wants to use S3 as storage, they can put Varnish in front of it and serve the data with less expense.

What do you lose out on? HTTP/2, SSL, etc. unless you put another HTTP server in front of varnish.

Re:How to Use Varnish Cache as Secured AWS S3 Gateway by andrewguenther on Friday February 19, 2016 @02:51PM (#11136091)
If you're comparing to Akamai, then shouldn't your data be in a CDN rather than S3?
Re:How to Use Varnish Cache as Secured AWS S3 Gateway by true_religion on Saturday February 20, 2016 @10:02AM (#11140279)
Not necessarily. I'm comparing bandwidth costs. Amazon S3 out-going bandwidth is an order of magnitude more expensive than some of its competitors (OVH for example), and, at least, 2x the price of Dreamhost's OpenStack solution (which includes a CDN for outgoing bandwidth).

Amazon S3 is essentially more expensive than a CDN for outgoing bandwidth even though it has only one PoP by default.

That's surprising since the typical case was that the CDN would be more expensive, and increasing the # of PoP you used in CDN would make it more expensive (e.g. there's a huge jump in pricing if you use Asia/Australia nodes).

Re:How to Use Varnish Cache as Secured AWS S3 Gateway by reza_n on Friday February 19, 2016 @01:46PM (#11135563)
So I wrote that blog post, was intended to be a quick little guide for S3 integration. As for your point regarding caching and bandwidth utilization, this actually comes up a bit, so its a fairly valid point. A lot of times software (and people) are accessing the same S3 asset, multiple times, all on the same network. So by going thru a local cache, you bypass having to reach all the way back to S3.
Re:How to Use Varnish Cache as Secured AWS S3 Gateway by coleifer on Friday February 19, 2016 @04:54PM (#11136932)
Lame of you not to link my post from yesterday about putting nginx in front of s3. This post seems clearly to be in response to mine.

http://charlesleifer.com/blog/nginx-a-caching-thumbnailing-r...

Re:How to Use Varnish Cache as Secured AWS S3 Gateway by fgsch on Friday February 19, 2016 @01:56PM (#11135645)
Also S3 has different costs depending on number of requests and transfer.
Re:How to Use Varnish Cache as Secured AWS S3 Gateway by andrewguenther on Friday February 19, 2016 @02:49PM (#11136076)
That use case makes a lot of sense. Thanks for the response!
Re:How to Use Varnish Cache as Secured AWS S3 Gateway by babo on Friday February 19, 2016 @04:28PM (#11136761)
and it gives you logging as well