News for hackers. Stuff that matters.
Re:AWS Lambda Edge changes duration billing granularity from... by isoprophlex on Friday April 09, 2021 @04:01PM (#26755357)
> Lightweight functions such as header manipulations or URL rewrites tend to have a short duration

I'm not knowledgeable AT ALL in this area, having never used or first-hand seen these FAAS things.

But header manipulation? Really... These are executed as aws lambdas?! I don't get why you'd ever need to do this.

(Not being facetious, honest puzzlement!)

Re:AWS Lambda Edge changes duration billing granularity from... by cactus2093 on Friday April 09, 2021 @09:13PM (#26758056)
It's funny our team literally just last week did something like this at ___ BigCorp. We need to verify a JWT signature from a new 3rd party provider and then append a secure auth token that is accepted by the backend service. Couldn't this just be done in the backend service? Why yes it definitely could. But apparently there are rigid security policies in place, and this stuff appears to have been carved in stone and approved by SOC2 auditors and lawyers and regulators or whatever and they have agreed that the backend gateway would use X specific formats for authentication and nobody can make any exceptions. So instead our team is spinning up an ad-hoc lambda function attached to cloudfront which will use none of our usual tooling & frameworks and have zero visibility for our security teams and be full of whatever soon to be out of date 3rd party js dependencies we need, all in the name of security so that we can pretend that the API we're communicating with is using our approved auth token format when they aren't. Just another day at a big company, really.
Re:AWS Lambda Edge changes duration billing granularity from... by rtpg on Friday April 09, 2021 @09:38PM (#26758182)
I mean a less cynical reading is that this is like nginx handling https->wsgi for your Python application. You can deal with a unification of your auth stack and not have application development be disrupted
Re:AWS Lambda Edge changes duration billing granularity from... by FpUser on Saturday April 10, 2021 @12:42AM (#26758988)
I would not call it cynical. Rather naming things for what they are.
Re:AWS Lambda Edge changes duration billing granularity from... by isoprophlex on Saturday April 10, 2021 @02:16AM (#26759364)
Interesting, and that actually makes (some sort of) sense. Thanks..!
Re:AWS Lambda Edge changes duration billing granularity from... by ryanmarsh on Friday April 09, 2021 @10:15PM (#26758376)
Oh hi. I see you work for one of my clients.
Re:AWS Lambda Edge changes duration billing granularity from... by gehen88 on Friday April 09, 2021 @04:40PM (#26755843)
We use Lambda@Edge to build our own CDN on top of S3 with authentication, so our customers have reliable, fast and secure data access. We use a bunch of edge lambdas which serve thousands of requests each minute, so I suspect we'll see a nice cost reduction with this.

Some of the stuff we do: - rewrite urls - auth and permissions check - serve index.html on directory indexes - inject dynamic content in served html files - add CORS headers

Re:AWS Lambda Edge changes duration billing granularity from... by nihakue on Friday April 09, 2021 @07:12PM (#26757319)
Can I ask how you're injecting dynamic content in served html files? My understanding was that lambda@edge functions could not modify the origin response body.

> When you’re working with the HTTP response, Lambda@Edge does not expose the body that is returned by the origin server to the origin-response trigger. You can generate a static content body by setting it to the desired value, or remove the body inside the function by setting the value to be empty. If you don’t update the body field in your function, the original body returned by the origin server is returned back to viewer.

https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope...

Re:AWS Lambda Edge changes duration billing granularity from... by heyoni on Friday April 09, 2021 @09:56PM (#26758283)
It sounds to me like they might be creating the response from scratch rather than modifying an existing one. And since they’re talking about serving data from s3, maybe it’s just a nicely formatted version of aws s3 ls?
Re:AWS Lambda Edge changes duration billing granularity from... by gehen88 on Saturday April 10, 2021 @02:30AM (#26759424)
We use S3.getObject to fetch it and create a custom response.
Re:AWS Lambda Edge changes duration billing granularity from... by pier25 on Friday April 09, 2021 @07:21PM (#26757389)
Isn't it expensive in terms of egress?
Re:AWS Lambda Edge changes duration billing granularity from... by donavanm on Friday April 09, 2021 @10:48PM (#26758524)
I'm not sure what you're specifically referring to by egress, but data transfer from S3/EC2/ELB etc to CloudFront is free:

> If you are using an AWS origin, effective December 1, 2014, data transferred from origin to edge locations (Amazon CloudFront "origin fetches") will be free of charge.

And in general data out from CloudFront ($0.085 to $0.02 in NA & EU) is cheaper than data out from S3 ($0.09 to $0.05 in us-east-2). For the lambda@Edge portion it's $0.60 per 1 million requests while the CloudFront requests are $0.75 to $1.00 per 1 million requests.

Those are public on demand rates and any customers with significant or consistent usage should definitely contact AWS for lower committed/custom prices.

Edit: Disclaimer I'm a principal at AWS, and in the past spent significant time working on CloudFront & Lambda@Edge, but all of the above is public information.

Re:AWS Lambda Edge changes duration billing granularity from... by diroussel on Friday April 09, 2021 @05:03PM (#26756139)
In AWS on the whole if there is any type of extension mechanism it's done via a lambda. Anything that needs an eval(), that is more than a regexp, is done via lambda.

This makes lambda's the glue of AWS. The input events are small json values, and the outputs are json values, and you can (mostly) use any language you want.

Re:AWS Lambda Edge changes duration billing granularity from... by jeffasinger on Friday April 09, 2021 @04:20PM (#26755581)
Essentially this is a way to lets you push all of the weird feature requests your customers might have off into one place.

From a customer point of view, maybe you need to rewrite the path based on the contents of a cookie, or maybe you want to shed certain types of requests in high load scenarios, or maybe there's a buggy upstream application that sends bad cache-control headers, or ...

If the goal is to let the customer specify infinitely complex logic at the edge, a programming language is a good way to do that. Function As A Service is a good billing model for lots of invocations of short, small functions across the customers choice of language.

Re:AWS Lambda Edge changes duration billing granularity from... by thayne on Friday April 09, 2021 @05:26PM (#26756396)
The unfortunate side effect is that Lambda@Edge at least adds quite a bit of latency that a simpler but less powerful rules based system for header manipulation probably wouldn't.

We experimented with using Lambda@Edge for manipulating a header, but the added latency just wasn't worth it, and we had to come up with a different solution. That was years ago though, so maybe it is better now.

Re:AWS Lambda Edge changes duration billing granularity from... by leesalminen on Friday April 09, 2021 @04:36PM (#26755798)
I use Lambda@Edge to conditionally serve brotli compressed .js files based on the Accept request header from an S3 bucket.
Re:AWS Lambda Edge changes duration billing granularity from... by alexbilbie on Saturday April 10, 2021 @01:06PM (#26763047)
Cloudfront now has native brotli support
Re:AWS Lambda Edge changes duration billing granularity from... by leesalminen on Saturday April 10, 2021 @03:52PM (#26764499)
That’s good to know, thanks!
Re:AWS Lambda Edge changes duration billing granularity from... by jdub on Saturday April 10, 2021 @01:03AM (#26759069)
It's especially useful for doing things during the Origin Request phase, which happens prior to the request from the CDN to the Origin.

In that phase you can change anything about the request itself, or the origin it will connect to. But the function only needs to run when there's a cache miss.

I have an application that needs to select an origin based on the request path (too many to use CloudFront cache behaviours), with a cache miss rate of less than 1%. It's a tiny amount of additional latency for cache misses, and a very small proportion of our CDN costs per month.

Re:AWS Lambda Edge changes duration billing granularity from... by adreamingsoul on Saturday April 10, 2021 @01:23AM (#26759138)
Varnish Cache with VCL allows you to create custom edge logic for things like header manipulation or backend selection.
Re:AWS Lambda Edge changes duration billing granularity from... by jdub on Saturday April 10, 2021 @03:24AM (#26759651)
Are you suggesting I build and maintain my own global CDN with 200+ edge locations? Thank you, but no thank you.
Re:AWS Lambda Edge changes duration billing granularity from... by jcpham2 on Saturday April 10, 2021 @07:34AM (#26760730)
I think so, as an alternative to all this AWS magic you could totally deploy <insert open source software name> on 200+ newly deployed instances, colo servers, or whatever you like!

I caught it too. I get downvoted a lot for not following rules and I don’t really care this was hilarious

Re:AWS Lambda Edge changes duration billing granularity from... by adreamingsoul on Sunday April 11, 2021 @02:31AM (#26767953)
cloud magic has shown us that “one day” we will need to scale up to an extreme count of “instances” and that suboptimal solutions can be negated with horizontal scaling.
Re:AWS Lambda Edge changes duration billing granularity from... by ts330 on Friday April 09, 2021 @04:22PM (#26755605)
basic manipulation in CF is an absolute pita - so much overhead for simple things like clobbering a header value.

but by allowing JS/whatever, it gives you an infinite landscape to work with the values in a language that a wide range of people grok.

it's just a matter of getting a decent deployment setup so you can rinse and repeat easily.

but i agree, it's a freaking jack hammer where most people need a small tack hammer.

Re:AWS Lambda Edge changes duration billing granularity from... by hardstyle on Saturday April 10, 2021 @12:41AM (#26758979)
We use Lambda@Edge so we can access Cloudfront’s true client IP value as well as geo data and pass it on to the origin
Re:AWS Lambda Edge changes duration billing granularity from... by singlow on Saturday April 10, 2021 @03:39AM (#26759717)
But cloudfront already sends that original ip back in a standard header.
Re:AWS Lambda Edge changes duration billing granularity from... by forty on Saturday April 10, 2021 @04:45AM (#26759998)
Yeah and you can ask Cloudfront to add some geodata as well (with headers such as CloudFront-Viewer-Country, etc)
Re:AWS Lambda Edge changes duration billing granularity from... by aeyes on Friday April 09, 2021 @04:58PM (#26756077)
> These are executed as aws lambdas

These are executed. Lambda is just a brand name. We did this 15 years ago on hardware load balancers, proxy servers, cache servers etc

Re:AWS Lambda Edge changes duration billing granularity from... by turtlebits on Friday April 09, 2021 @04:12PM (#26755492)
What would you propose? Yet another AWS service for the specific use case?
Re:AWS Lambda Edge changes duration billing granularity from... by luhn on Friday April 09, 2021 @06:15PM (#26756860)
CloudFront should let you configure header injection for responses. Just a simple list of key-value pairs. I've seen it requested before, it's a feature a lot of people want.
Re:AWS Lambda Edge changes duration billing granularity from... by hamandcheese on Friday April 09, 2021 @04:18PM (#26755559)
Config of some sort maybe? If you can do it in nginx, seems like it should be possible without invoking a lambda.
Re:AWS Lambda Edge changes duration billing granularity from... by njitbew on Friday April 09, 2021 @04:47PM (#26755939)
I don't understand what nginx has to do with this. nginx's language is just that: a language. If Lambda@Edge would support nginx language, you would still be invoking a lambda. But I'd must rather write some header rewriting code in, say, JavaScript than nginx's language.

Then why a (Turing-complete) programming language and not a config language? Simple: because a programming language gives you infinite more power than what any config system could ever provide. E.g. I could compare the origin IP, determine the country, and change the response based on that. And there are plenty of use cases like this which you cannot express in a config language.

Re:AWS Lambda Edge changes duration billing granularity from... by jfrunyon on Friday April 09, 2021 @05:17PM (#26756295)
> E.g. I could compare the origin IP, determine the country, and change the response based on that. And there are plenty of use cases like this which you cannot express in a config language.

Well, you can do that in plenty of "config languages", including nginx[1]. The reason Lambda exists is that it's good for Amazon, because they can push all the costs of the logic back onto the customer. (It could also be good for the customer, in that whatever configuration Amazon might otherwise support could be limited.)

[1] http://nginx.org/en/docs/http/ngx_http_geoip_module.html

Re:AWS Lambda Edge changes duration billing granularity from... by TheCoelacanth on Friday April 09, 2021 @05:33PM (#26756483)
If you want a full-fledged reverse proxy, then you can run a full-fledged reverse proxy.

Cloudfront is a CDN, it's not intended to be a replacement for nginx.

Re:AWS Lambda Edge changes duration billing granularity from... by kjjjjjjjjjjjjjj on Friday April 09, 2021 @07:08PM (#26757293)
If you have stuff in a global CDN, wouldn't you want a global compute system too? This is literally what lambda@edge is for