Re:Lambda@Edge – Preview
by scrollaway on Thursday December 01, 2016 @05:20PM
(#13084103)
So we're getting C# and a new Node.js runtime for Lambda. Jeff, surely you're reading this - when are we getting Python 3 support as an additional runtime?
I've been waiting for months for re:invent in the hope that it would be announced today, only to be disappointed. I'm not the only one in this situation: stuck on Python 2.7, hitting issues left and right because of an ancient runtime that we no longer would care about were it not for Lambda.
Just today I hit another issue in production which impacted our webhook system. Wouldn't you know it, `json.dumps()` produces illegal json in Python 2.7 when you give it an IntEnum:
>>> from enum import IntEnum; from json import dumps, loads
>>> class E(IntEnum):
... A = 1
>>> dumps(E.A)
'E.A'
>>> loads(dumps(E.A))
Traceback (most recent call last):
...
ValueError: No JSON object could be decoded
Please. Do something about this. It's wasting so much time :(
Re:Lambda@Edge – Preview
by Mizza on Thursday December 01, 2016 @05:24PM
(#13084130)
Was really expecting a lot more out of the announcements today as well. Python3, GPUs, custom containers, larger packages.. the list goes on..
That being said, re: Python 3, you should keep a close eye on Zappa (https://github.com/Miserlou/Zappa) - as we've got a surprise in store that may delight you!
Re:Lambda@Edge – Preview
by movedx on Thursday December 01, 2016 @06:04PM
(#13084450)
So this looks very cool. Great job... but...
It would be pretty simple to wrap this up and automate it easily using a Terraform file. All it requires is an S3 resource for the ZIP, a Lambda resource using the S3 resource for the code, and finally an API Gateway resource to tie it all together.
Re:Lambda@Edge – Preview
by Mizza on Thursday December 01, 2016 @06:13PM
(#13084524)
...but why?
We already use CloudFormation under the hood for the API Gateway, though to be honest I'm seriously considering moving away from it because it's so fickle.
There are also some security disadvantages and other problems from using CloudFormation for maintaining Lambda functions, plus you lose out on all the extensive functionality we provide. We're pretty damn well battle tested to avoid that stuff now.
Re:Lambda@Edge – Preview
by movedx on Friday December 02, 2016 @06:27AM
(#13087756)
Well, by using Terraform you're opening up the door to using other Lambda offerings as they pop up from other venders (and become supported by Terraform, of course.) CloudFormation is nice, and in your case it makes perfect sense given you're offering a Lambda specific service, but in the grand scheme of things I don't believe it's the best solution "all round."
What you're offering is awesome, for the record. I think it's a good idea, I think it's powerful, I think it's fast. It's a good idea and you should keep working on it.
However, it's a single, small Terraform file worth of work from my perspective.
Re:Lambda@Edge – Preview
by scrollaway on Thursday December 01, 2016 @05:32PM
(#13084196)
I've looked at Zappa in the past; it's very impressive, but it's overkill for what we're doing - we're only processing files and running a Django ORM instance behind it. And because we have quite a high throughput, running a Python 3 shim behind Python 2 would increase our costs by too much.
Thank you for making it though. It's a really cool part of the Lambda ecosystem. Excited to hear there's something in store for Py3 :)
Re:Lambda@Edge – Preview
by kiwidrew on Thursday December 01, 2016 @05:16PM
(#13084075)
This is very, very cool. It allows you to deploy application logic extremely close to the user (in terms of latency), and Cloudfront has an order of magnitude more edge locations than the other AWS services like EC2 and S3.
In fact, if the promised ability to read/write the request/response bodies does appear, it might be possible to deploy your entire website or app as a single Lambda@Edge function. (Not every app would fit within the size, memory, and CPU constraints, of course.)
The main limitation would probably be the fact that for each incoming HTTP request, your choices are limited to either making zero origin requests (i.e. the function generates the response directly) or one origin request. It's not clear how this could work if you wanted to make two origin requests and combine them to render a single response.
I'm totally going to sign up for the preview and play around with it! Especially if, like the regular node.js Lambda functions, you can include arbitrary Linux binaries with the function code. Compiled request-processing code would easily meet the 128MB/50ms criteria.
This should definitely meet our needs in both cases: serving prerendered content when necessary, and also adding the security headers to the front end. And it's definitely a 10x better solution than having an extra EC2 instance with your private certs on it, just to run an nginx proxy with twenty lines of code.
Re:Lambda@Edge – Preview
by apendleton on Thursday December 01, 2016 @06:07PM
(#13084473)
It's not supported in IE 6 on any OS, or any IE version on Windows XP or before (has to be supported by both the browser and the underlying network stack). Also not supported on Android < 3.x.
Re:Lambda@Edge – Preview
by shade23 on Thursday December 01, 2016 @02:42PM
(#13082738)
>Without spilling any beans I let him know that we were very aware of this use case and that we had some interesting solutions in the works. Other customers have also let us know that they want to customize their end user experience by making quick decisions out at the edge.
Re:Lambda@Edge – Preview
by saurik on Thursday December 01, 2016 @02:44PM
(#13082759)
Ah, OK: the answer is "no" (as the question did not lead to the feature), not "yes" ;P; and also "I am dumb" (FWIW, I actually got to this thread by searching for it after already knowing about @Edge, so I didn't even click the article link: I saw the @Edge stuff today in the ECS FAQ, searched for it on Google and saw it had been announced 15 minutes earlier, and then came to HN to read comments about it).
Re:Lambda@Edge – Preview
by saurik on Thursday December 01, 2016 @02:09PM
(#13082422)
While having minimal access to the request to do things like add headers is of course quite useful, the people I've known who most want code execution at the edge are either already using or looking at using ESI (Edge-Side Includes, as provided by Akamai and Varnish), in order to express things like "use the cached content for almost the entire page but add some per-user information to the response via a secondary virtual request (the content for which might also be cached at the edge!) for people who are logged on; they say that at some point they will allow access to and potentially modification of the body, but it isn't clear to me that this will be as potent as (even if it eventually becomes technically as powerful as) ESI.
Re:Lambda@Edge – Preview
by hackcrafter on Thursday December 01, 2016 @02:08PM
(#13082412)
> Because your JavaScript code will be part of the request/response path, it must be lean, mean, and self-contained. It cannot make calls to other web services and it cannot access other AWS resources. It must run within 128 MB of memory, and complete within 50 ms.
I'm having a hard time conceptualizing what kind of logic you would run on these edge functions?
You have access to headers, so also things like client cookies, so could you use that to potentially personalize/templatize the body (once they have body mutation abilities)?
Or is it really just about static content routing? Detect google bot user agent and send them to resource A versus B etc...
Re:Lambda@Edge – Preview
by Alex3917 on Thursday December 01, 2016 @02:36PM
(#13082670)
> I'm having a hard time conceptualizing what kind of logic you would run on these edge functions?
Two use cases:
1) Routing Facebook and Twitter to a pre-rendered version of your single page app based on their user agents.
2) Adding custom headers to your responses, e.g. for a Content Security Policy.
There is no good way to do either of those things currently. You could have one off options for each of those and the 50 other common use cases, but this is actually a really elegant solution. Very well done.
Re:Lambda@Edge – Preview
by jalk on Thursday December 01, 2016 @04:12PM
(#13083544)
This is totally cool if the origin is just a dumb S3 bucket. From what I can tell, you can implement the same kind of rewrite logic you would otherwise have in i.e. httpd/nginx conf.
And of course doing so at the edge means that you potentially shave off some latency during redirects.
Re:Lambda@Edge – Preview
by wiredfool on Thursday December 01, 2016 @04:12PM
(#13083550)
It's a potentially cleaner way to send back a redirect from API gateway/lambda rather than pattern matching on http in the error string of the Jason response body.
Re:Lambda@Edge – Preview
by mifreewil on Thursday December 01, 2016 @02:23PM
(#13082544)
The linked post uses CloudFront as an example, which is commonly used simply as a CDN in front of static content on S3. This allows you to take that model and add some dynamic content to it.
I just recently used this model to host a JS widget used on external websites, now I could for example add some basic logic to disallow certain domains (by checking referrer headers) from using the widget.
Re:Lambda@Edge – Preview
by forrestbrazeal on Thursday December 01, 2016 @02:11PM
(#13082433)
The blog post says that one of the primary use cases is to help preview content show up correctly on Facebook, etc. So I don't think client-side routing for static apps will work in this case.
Re:Lambda@Edge – Preview
by jotto on Thursday December 01, 2016 @04:25PM
(#13083662)
For anyone needing a minimal effort prerendering solution (server side render for most JS frameworks), i.e. the use case described by Jeff - I made https://www.prerender.cloud/
(you could use the prerendercloud npm middleware from the lambda edge)
Re:Lambda@Edge – Preview
by dkordik on Thursday December 01, 2016 @04:32PM
(#13083727)
Interesting... looks like something we might want to use to do URL rewriting for an S3-hosted SPA test environment, so that deep links resolve to the base of the SPA without having to use a more "dynamic" host like an EC2 instance running nginx. [edit] ah, nevermind, you have to be using CloudFront too.
Re:Lambda@Edge – Preview
by brutopia on Thursday December 01, 2016 @06:51PM
(#13084725)
This so much! Cloudfront/lambda/s3 combo is missing this piece for which you have to use unnecessarily complex api gateway or ec2 varnish/nginx setup. Thanks Jeff!
Re:Lambda@Edge – Preview
by slaymaker1907 on Thursday December 01, 2016 @04:18PM
(#13083592)
I somewhat wonder if this will be used by malicious websites for cloaking. For those who are unaware of the term, cloaking is used by malicious websites/ads to render different content to search engines and bots than for real users to avoid detection by those automated agents.
Re:Lambda@Edge – Preview
by pedalpete on Thursday December 01, 2016 @05:11PM
(#13084057)
This seems it might be a way to check if a user is logged-in or not and provide the appropriate page. I'm thinking if the user details are in a jwt or cookie, you could render the correct page on insecure routes.
Re:Lambda@Edge – Preview
by dustingetz on Thursday December 01, 2016 @04:23PM
(#13083641)
you can use this to serve dynamic websites out of CDN, for example github pages, which we talk about as a static site but underneath it is a slow moving database (your git repo)
I've been waiting for months for re:invent in the hope that it would be announced today, only to be disappointed. I'm not the only one in this situation: stuck on Python 2.7, hitting issues left and right because of an ancient runtime that we no longer would care about were it not for Lambda.
Just today I hit another issue in production which impacted our webhook system. Wouldn't you know it, `json.dumps()` produces illegal json in Python 2.7 when you give it an IntEnum:
Please. Do something about this. It's wasting so much time :(That being said, re: Python 3, you should keep a close eye on Zappa (https://github.com/Miserlou/Zappa) - as we've got a surprise in store that may delight you!
It would be pretty simple to wrap this up and automate it easily using a Terraform file. All it requires is an S3 resource for the ZIP, a Lambda resource using the S3 resource for the code, and finally an API Gateway resource to tie it all together.
Thoughts?
We already use CloudFormation under the hood for the API Gateway, though to be honest I'm seriously considering moving away from it because it's so fickle.
There are also some security disadvantages and other problems from using CloudFormation for maintaining Lambda functions, plus you lose out on all the extensive functionality we provide. We're pretty damn well battle tested to avoid that stuff now.
What you're offering is awesome, for the record. I think it's a good idea, I think it's powerful, I think it's fast. It's a good idea and you should keep working on it.
However, it's a single, small Terraform file worth of work from my perspective.
Thank you for making it though. It's a really cool part of the Lambda ecosystem. Excited to hear there's something in store for Py3 :)
Looks like you have to jump through a couple of hoops but that might work for running Python 3.
https://news.ycombinator.com/item?id=13084196
In fact, if the promised ability to read/write the request/response bodies does appear, it might be possible to deploy your entire website or app as a single Lambda@Edge function. (Not every app would fit within the size, memory, and CPU constraints, of course.)
The main limitation would probably be the fact that for each incoming HTTP request, your choices are limited to either making zero origin requests (i.e. the function generates the response directly) or one origin request. It's not clear how this could work if you wanted to make two origin requests and combine them to render a single response.
I'm totally going to sign up for the preview and play around with it! Especially if, like the regular node.js Lambda functions, you can include arbitrary Linux binaries with the function code. Compiled request-processing code would easily meet the 128MB/50ms criteria.
https://news.ycombinator.com/item?id=13011120
This should definitely meet our needs in both cases: serving prerendered content when necessary, and also adding the security headers to the front end. And it's definitely a 10x better solution than having an extra EC2 instance with your private certs on it, just to run an nginx proxy with twenty lines of code.
It was in the pipeline already I guess.
I'm having a hard time conceptualizing what kind of logic you would run on these edge functions?
You have access to headers, so also things like client cookies, so could you use that to potentially personalize/templatize the body (once they have body mutation abilities)?
Or is it really just about static content routing? Detect google bot user agent and send them to resource A versus B etc...
Two use cases:
1) Routing Facebook and Twitter to a pre-rendered version of your single page app based on their user agents.
2) Adding custom headers to your responses, e.g. for a Content Security Policy.
There is no good way to do either of those things currently. You could have one off options for each of those and the 50 other common use cases, but this is actually a really elegant solution. Very well done.
And of course doing so at the edge means that you potentially shave off some latency during redirects.
I just recently used this model to host a JS widget used on external websites, now I could for example add some basic logic to disallow certain domains (by checking referrer headers) from using the widget.
(you could use the prerendercloud npm middleware from the lambda edge)