News for hackers. Stuff that matters.
Re:Web Scraping 101 with Python by rinze on Wednesday February 10, 2021 @07:48PM (#26096866)
Not my project, but if I had to do it I'd try something like the following:

* Set an autoscaling group with your instance template, max instances 1, min instances 0, desired instances 0 (nothing is running).

* Set up a Lambda function that sets the autoscaling group desired instances to 1.

* Link that function to an API Gateway call, give it an auth key, etc.

* From any machine you have, set up your cron with a random sleep and a curl call to the API.

And that should do the trick, I think.

Re:Web Scraping 101 with Python by ses1984 on Wednesday February 10, 2021 @11:02PM (#26098125)
>From any machine you have, set up your cron with a random sleep and a curl call to the API.

You might as well just call the ASG API directly.

Re:Web Scraping 101 with Python by warsheep on Thursday February 11, 2021 @01:37AM (#26098888)
Why use autoscaling and not just launch the instance directly from lambda? The run time is short so there's no danger of two instances running in parallel