Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on SpiderKeeper: Admin UI for scrapy/open source scrapinghub. Parent comment
Airflow is for hooking together and managing a bunch of data sources, data-processing nodes, and data sinks.
This project seems like a GUI for Scrapy, which is a web crawling framework, i.e. one kind of data source.
There are no comments on this comment yet.
Airflow is for hooking together and managing a bunch of data sources, data-processing nodes, and data sinks.
This project seems like a GUI for Scrapy, which is a web crawling framework, i.e. one kind of data source.