At first I wanted to put my scripts in Lambda, put an API Gateway in front, and have them be callable by the Google Sheet on a schedule. This seemed "pure" because I didn't really _need_ independent resources to do the task. The sheet could just update itself using an on-demand server!
Then I realized there was a 30-second max execution time. My scripts, in serial, couldn't hit that. I'd need to rewrite them all to be parallel. Also, they were a bunch of bash/jq/curl I'd just thrown together. I'd have to write a Python wrapper to shell out and handle streams (ugh).
At this point, I gave up and fell back to the advice of an (ex-Heroku) coworker: why not just run them in a cron job and put the data in a Heroku DB and use Heroku Dataclips to update the Google Sheet.
Sure I have to "have" an EC2 instance and a Heroku PG box, but they're both so small they're either free or essentially so. After cloning my script repos, everything worked right away.
Sometimes it's better just to do the "inelegant" thing and move on.
Perhaps you're talking about a limitation on your spreadsheet, but the max execution time for a Lambda function is configurable up to 300 seconds.
> 30 seconds for all integration types, including Lambda, Lambda proxy, HTTP, HTTP proxy, and AWS integrations.
So not Lambda alone, but in conjunction with the API Gateway.
I didn't have to worry about setting up automation for launching an ec2 instance, monitoring it, paying for it, or any of the complex trouble for what ended up being a single python function that cost abotu $0.10 per month to run.
You're right that there are other straightforward options, but they do have downsides. Worst case is you have mismatch of workloads (e.g. a rogue cronjob affecting another service). Or you end up with resources that are largely idle.
That said, I agree with your unease on the application side. If you had a dozen (or many dozen) of these you've got pieces and parcels of logic all over the place. I think some architecture/style changes could be mitigations here. A shared library might be one.
> The cron job that we needed to write was unrelated to our application code, so whilst we could have put the functionality in there it seemed like the wrong place.
he never looked back at existing architecture and invented a challenge for himself.
I really feel like the author boxed himself into a solution by faulty reasoning.
1) Cron jobs are not hard to set up. Being able to control these things on one or more server is just part of proper server deployment.
2) If it is not part of the application but seems to have to do with information about users who used the application in trial then maybe it should be part of the application? If you are deploying a application and you require to know information about users who trials are going to end I sure would bet that is part of the application and should be a trigger coded into the application itself.
3) I am shocked that you are already did not have a central spot for running time interval code and maintenance on your systems. At a minimum there should have been backups and reports generated on your mongodb instances. If these functions were already built into the application then then the same generic task system you used to run that is likely where you wanted to run this new functionality.
tl;dr Not knowing much about the application or its architecture I feel like the author painted themselves into a corner to justify using some AWS tools.