News for hackers. Stuff that matters.
Re:Show HN: Crawlee – Web scraping and browser automation li... by OlikEten on Tuesday August 23, 2022 @04:24AM (#32561982)
Nice! Finally a web scraping library for the programming language most websites use. It was about time lol
Re:Show HN: Crawlee – Web scraping and browser automation li... by mvolfik on Tuesday August 23, 2022 @05:58AM (#32562544)
Thank you! And exactly as you say, using the same language as the websites gives us some advantages - we have HTTP-only (no client-side JS) crawlers based on the Cheerio library, which mimics jQuery API, and if you later find out that you need to use a full headless browser with Puppeteer, you can just call the utility function injectJquery, and there's very little you have to modify to keep your script working