I do a lot of scraping at my day job, and I agree the times we need to use a headless browser are very rare. Vast majority of the time you can either find an API or JSON endpoint, or scrape the JS contained in the returned SSR document to get what you want.