News for hackers. Stuff that matters.
Re:Monoliths Are the Future by ForHackernews on Thursday January 30, 2020 @03:08PM (#22194967)
What/where do you run this mythical one-analyst pipeline, though? Is that in cloud services too? Airflow? Kubeflow? Apache Beam? It sounds like you're just pushing the problem around.
Re:Monoliths Are the Future by WoahNoun on Thursday January 30, 2020 @03:16PM (#22195045)
AWS data pipelines and AWS lambda. It's cloud services the whole way down.

https://aws.amazon.com/datapipeline/

Re:Monoliths Are the Future by geggam on Thursday January 30, 2020 @04:14PM (#22195678)
I saved a company 20k a month by creating a job server in AWS. Lambda isnt cheap when you start using it hard
Re:Monoliths Are the Future by WoahNoun on Thursday January 30, 2020 @05:12PM (#22196360)
Lambda is mostly used for it's trigger functionality for data or artifacts that are created at irregular intervals. Eg, an object is uploaded to s3 which triggers a lambda which runs a copy command for that object into redshift. The kind of stuff that's well below the threshold for leaving the free tier.