Web Crawler and Scraper for AI

Posted by catskindleyou on Thursday December 19, 2024 @02:39PM
from the spider.cloud dept.

( 4 comments | 4 points | on HN )

Re:Web Crawler and Scraper for AI by adhamsalama on Thursday December 19, 2024 @03:05PM (#42465185)

What's with web scraping tools rebranding as "web scraping for AI"? It has nothing to do with AI at all.

[ Parent | on HN ]

Re:Web Crawler and Scraper for AI by catskindleyou on Thursday December 19, 2024 @07:01PM (#42467051)

AI tools need to get the data in the proper formats while not letting the data layer being a bottleneck.

[ Parent | on HN ]

Re:Web Crawler and Scraper for AI by whatnotests2 on Thursday December 19, 2024 @05:15PM (#42466282)

Very cool - and impressive speed!

How does it compare with headless browser services like https://browserless.io or https://browserbase.com ?

[ Parent | on HN ]

Re:Web Crawler and Scraper for AI by catskindleyou on Thursday December 19, 2024 @06:57PM (#42467022)

The chrome speed is also holding the bar high processing thousands of pages per second.

[ Parent | on HN ]