Re:Lessons learned from running Apache Airflow at scale
by idleprocess on Monday May 23, 2022 @01:00PM
(#31481648)
I can second this. We were up-and-running with Docker on our dev machines in just a few minutes. A native installation involves substantially more setup (Python, databases, Redis and/or Rabbit, etc.). The published docker-compose file will handle all of that for you. We have a very small data engineering team and have been able to move very quickly with Docker and AWS ECS (for orchestrating containers in test and prod environments).
[ Parent
| on HN ]
There are no comments on this comment yet.