News for hackers. Stuff that matters.
Re:Airflow and the Future of Data Engineering: A Q&A by thibaut_barrere on Wednesday March 01, 2017 @03:33AM (#13762250)
If you are also using Ruby, you can use Kiba ETL (disclaimer: I'm Kiba ETL author) in conjunction with Sidekiq (quick example: https://gist.github.com/thbar/ec90bbd5877d1aae40510fadd43206...).

This allows to build ETL jobs which react to all sorts of external application triggers (such as an upload made by a user in that case, but it could be an API notification / webhook etc).

There are no comments on this comment yet.