News for hackers. Stuff that matters.
Re:Ask HN: How was your experience with AWS Lambda in produc... by callumlocke on Wednesday June 21, 2017 @04:58AM (#14602105)
I made an image hosting tool on Lambda and S3, for internal corporate use. Staff can upload images to S3 via an SPA. The front end contacts the Lambda service to request a pre-signed S3 upload URL, so the browser can upload directly to S3. It works really well. Observations:

1. Took too long to get something working. The common use case of hooking up a Lambda function to an HTTP endpoint is surprisingly fiddly and manual.

2. Very painful logging/monitoring.

3. The Node.js version of Lambda has a weird and ugly API that feels like it was designed by a comittee with little knowledge of Node.js idioms.

4. The Serverless framework produces a huge bundle unless you spend a lot of effort optimising it. It's also very slow to deploy incremental changes edit: – this is not only due to the large bundle size but also due to having to re-up the whole generated CloudFormation stack for most updates.

5. It was worth it in the end for making a useful little service that will exist forever with ultra-low running costs, but the developer experience could have been miles better, and I wouldn't want to have to work on that codebase again.

---

Edit: here's the code: https://github.com/Financial-Times/ig-images-backend

To address point 3 above, I wrote a wrapper function (in src/index.js) so I could write each HTTP Lambda endpoint as a straight async function that simply receives a single argument (the request event) and asynchronously returns the complete HTTP response. This wouldn't be good if you were returning a large response though; you'd probably be better streaming it.

Re:Ask HN: How was your experience with AWS Lambda in produc... by weeefun on Wednesday June 21, 2017 @07:48AM (#14602845)
Although I've just started using AWS Lambda and think it's a cool technology to use, I can understand your points.

2) Logging is indeed painful! You definitely need a separate tool/system for that. I have created a small CLI tool to view logs of multiple Lambdas which have been deployed using a CloudFormation template: https://github.com/seeebiii/lambdalogs This does not replace a good external system, but it can help for small searches in the logs.

4) Yes, this takes a lot. Though I'm not using the Serverless framework, deploying the code using CloudFormation takes me about 2:30 minutes (with a project using Java Lambdas), because CF is doing lots of checks in the background. I also wrote a tool for this to decrease the waiting time and just update the JS/Java code instead of the whole stack: https://github.com/seeebiii/lambda-updater

Hope this helps you or someone else a bit!

Re:Ask HN: How was your experience with AWS Lambda in produc... by mcrider on Wednesday June 21, 2017 @09:25AM (#14603530)
For those using Serverless, being able to upload just the code (not a full Cloudformation stack) is built in, and will save you tons of time. Also, do what you can to limit/optimize your npm modules (if you're using node) since it has to be uploaded every time.
Re:Ask HN: How was your experience with AWS Lambda in produc... by kesor on Wednesday June 21, 2017 @11:32AM (#14604644)
The problem of logs is actually a problem of CloudWatch Logs being just not a very good service. A great way to solve that is to push all logs from CloudWatch Logs into an ElasticSearch cluster (using a Lambda function). AWS even has the code already done for you if you click the "subscribe" button in CWL. Then with Kibana/ElasticSearch the experience of inspecting and analysing logs is MUCH better.
Re:Ask HN: How was your experience with AWS Lambda in produc... by jamroom on Wednesday June 21, 2017 @11:45AM (#14604777)
This is what drives me nuts about AWS, but is genius on their part - they've got you to spend more $$ on ElasticSearch and Lambda to overcome the fact that one of their other products just doesn't work very well.
Re:Ask HN: How was your experience with AWS Lambda in produc... by milesvp on Wednesday June 21, 2017 @08:23PM (#14609183)
I don't think this is intentional. My experience with AWS is that they aim to constantly improve products and create hosted services that are cheaper than if you frankensteined the same thing yourself. They don't always succeed, but I'm willing to give them the benefit of the doubt for how bad cloudwatch truly is, and just assume either they're blind to the pain because they know what not to do internally, or they're just really hamstrung in trying to modify that feature since so much relies on it.

Hopefully this isn't just stockholm syndrome speaking though...

Re:Ask HN: How was your experience with AWS Lambda in produc... by adjohn on Wednesday June 21, 2017 @04:12PM (#14607472)
We built IOpipe[1] to address these issues by offering our own wrapper[2] that sends telemetry to our service. IOpipe aggregates metrics, and errors, and allows the creation of alerts with multiple rules per alert.

[1] - https://iopipe.com [2] - https://github.com/iopipe/iopipe/

Re:Ask HN: How was your experience with AWS Lambda in produc... by munns on Thursday June 22, 2017 @09:12AM (#14611958)
+1 to the folks at IOpipe. A really cool product that gives you very interesting visibility into your Lambda function executions!
Re:Ask HN: How was your experience with AWS Lambda in produc... by OJFord on Sunday June 25, 2017 @06:30AM (#14629446)
FYI: except on very wide screens that landing page copy is barely readable over the background image, and with a narrow window it's also obscured by the header and navigation.
Re:Ask HN: How was your experience with AWS Lambda in produc... by lindydonna on Wednesday June 21, 2017 @07:08PM (#14608780)
Have you tried out Azure Functions? It has pretty good features for continuous integration, a CLI where you can run locally, and a really nice monitoring experience. Obviously it's probably not worth migrating an existing project, but it might be useful for future projects. Plus, we have a Serverless Framework plugin.

Here's a talk that walks through some of our features: https://www.youtube.com/watch?v=TgB-fs1hwlw&t=18s

Disclosure: I'm a Program Manager on Azure Functions.

Re:Ask HN: How was your experience with AWS Lambda in produc... by thenewwazoo on Wednesday June 21, 2017 @08:49PM (#14609311)
I used Azure Functions (when I was a MSFT employee, in fact), and found it to be unusable. A list of complaints:

* setup is 100% completely clicky-clicky UI driven, which was a huge pain to scale. instantiation of a Function on behalf of a developer for production use was a huge time sink

* it's clearly a thin veneer on Azure Web Services, and the abstractions leak badly in the portal (deployment credentials, for example)

* the web UI breaks completely and mysteriously if you enable authentication

* management of service princi- uh, I mean, Azure AD Applications was weird, and the (internal to MSFT, I suspect) permissions model to the Graph API was a huge barrier to ease of use

* management of NPM packages required me to start a terminal session in the UI and run commands manually, which was a huge turnoff (and had to be repeated ad nauseam with every new Function created)

* the configuration files for the runtime are utterly undocumented, with the sole exception of the bits used to plug Azure inputs/outputs together. this makes automating things exceedingly difficult. I recall there even being a magic value in the topmost config file

* the edit-commit-push-test cycle was VERY slow, with new commits sometimes taking tens of minutes to "appear" in my function

* I never found a way to run it locally, making the previous point that much worse

* log output is very difficult to find, and can live in a few different places. I spent too much time hunting for errors, especially things like syntax errors that make the runtime itself go kaboom. This was the thing that really killed it for me; if I had an error that resulted in anything but a "clean" return, it was torture trying to figure out where I'd missed the paren.

Re:Ask HN: How was your experience with AWS Lambda in produc... by lindydonna on Thursday June 22, 2017 @01:57PM (#14614041)
Thanks for the detailed feedback! I think you probably used Functions when it was much newer, and I think we’ve actually addressed all of your issues.

- You can create a Function App via ARM/CLI/etc., you can write functions without ever touching the portal. See https://docs.microsoft.com/en-us/azure/azure-functions/funct.... You can also now use Visual Studio to author C# functions: https://docs.microsoft.com/en-us/azure/azure-functions/funct...

- It’s true that Functions is built on App Service, but I see that as an advantage. You get all the great features of Continuous Integration, custom domains, automated deployment, etc.

- Indeed, the portal does not do well when auth is enabled and all routes are protected. The problem is that the portal calls admin APIs that are also protected, so it fails. We now have better error messages for this, and we’re tracking this bug: https://github.com/Azure/azure-functions-ux/issues/499

- The Graph API issue is probably not specific to Functions, but it is a bit easier with the Authentication/Authorization feature. Can you provide more detail?

- You can install npm packages at the "root" of your Function and not reinstall them for each Function, just like a normal Node.js app - it walks the directories.

- Our documentation is much better now, and we even have documentation for all bindings in the portal. We also have much better conceptual docs on bindings, see https://docs.microsoft.com/en-us/azure/azure-functions/funct.... We’d welcome any specific feedback on docs that are missing.

- CI should be faster now, it usually takes about 2-3 minutes for commits to show up. It’s fast enough that I’ve demo’d it.

- You can now run locally and debug using the Azure Functions Core Tools (npm i -g azure-functions-core-tools; func init; func host start). See docs: https://docs.microsoft.com/en-us/azure/azure-functions/funct.... This is something that our users always praise us for. We support C# debugging with Visual Studio and JavaScript debugging with VSCode.

- Logs definitely weren't great. Initially, they always went to table storage, but the ones you see streaming in the portal get written to disk to enable the realtime portal stream - they are only written to disk when you're in the portal, so they are "sometimes" there. The good news is that we've tightly integrated Application Insights, which means logs are easy to find. It's easy to alert on failed functions. You can see perf and metric data all in one place without log parsing. For a demo, go to the 6 minute mark of this video: https://www.youtube.com/watch?v=TgB-fs1hwlw&t=6m

Re:Ask HN: How was your experience with AWS Lambda in produc... by lindydonna on Friday June 23, 2017 @05:01PM (#14622757)
Update: the portal does work if Auth is enabled--the bug was fixed a couple of months ago and we didn't close the issue. See https://github.com/Azure/azure-functions-ux/issues/499
Re:Ask HN: How was your experience with AWS Lambda in produc... by thenewwazoo on Sunday June 25, 2017 @04:54PM (#14632123)
Thanks for such a comprehensive follow-up. It's true that it has been a while, and it really does sound like you've addressed nearly all of the issues I had. (btw, the undocumented file was host.json, which it appears may be better documented now)
Re:Ask HN: How was your experience with AWS Lambda in produc... by brango on Wednesday June 21, 2017 @08:31AM (#14603134)
Chalice [1] makes hooking up lambda to HTTP endpoints easy, provided you don't mind using AWS API Gateway (and using python).

I use it to handle a contact form on a static web site. It works really well.

[1] https://github.com/awslabs/chalice

Re:Ask HN: How was your experience with AWS Lambda in produc... by Prefinem on Wednesday June 21, 2017 @02:10PM (#14606395)
1. I think a framework / tool will ensure this isn't an issue

2. Dear heavens yes. I ended up building a wrapper (similar to what you did to address 3) that handles logging, etc for any internal events. Everything else is a pass / fail check

3. Also had to build a wrapper. Context / Callback params are confusing

4. I wouldn't use Serverless unless you need it. Try something smaller. Apex is a nice, simple start. Shameless plug: I built a deployment tool because serverless wouldn't work for us and I wanted something in node (no binary like Apex - integrate into our build process) [1]

[1]: https://github.com/Prefinem/lambdify

Re:Ask HN: How was your experience with AWS Lambda in produc... by z3t4 on Wednesday June 21, 2017 @02:01PM (#14606307)
how do you make sure it will run forever ? what about breaking changes and discontinued services ? can you move to another vendor without changes to your codebase ?
Re:Ask HN: How was your experience with AWS Lambda in produc... by 2pointsomone on Wednesday June 21, 2017 @05:36AM (#14602312)
Ditto.