From a naive point of view it's easy. But it has to scale and launching a new process for every request works only for a pet project.

So a crawler based on a headless browser that consumes little memory and runs for weeks is a major achivement.