We’re using Apify to scrape with their Web Content Scraper actor. It’s nice because they handle proxying for us which decreases the likelihood of us getting blocked. Their “to markdown” functionality is also a lot better than libs you find on npm. They are expensive tho