Any plans for more articles, 10 years later?

If I would like to build a visualization like this, but for a data ingestion pipeline, any tips on where to start?

I have it visually in my head, but it feels overwhelming getting it into a website.

fwiw I work on data ingestion pipelines and I've found that starting with just boxes-and-arrows in something like Excalidraw gets you 80% of the way to knowing what you actually want. The gap between "I can picture it" and "I can build it on a webpage" is mostly a d3 learning curve problem, not a design problem.

xyflow that the creator mentioned is probably the right call for pipeline DAGs though -- we use it internally for visualizing our scraping workflows and it was surprisingly painless to get running

Sort of like this? https://docs.tecton.ai/docs/introduction/interactive-tour I used https://github.com/xyflow/xyflow for this, with css animations for the edges. It’s probably easier now with coding agents and what not