A web scraping CLI made for AI that is idempotent

Re:A web scraping CLI made for AI that is idempotent by katella on Friday August 16, 2024 @07:31PM (#41271281)

Does it just scrape all pages of a site?

[ Parent | on HN ]

Re:A web scraping CLI made for AI that is idempotent by clemlesne on Saturday August 17, 2024 @02:46PM (#41276900)

It saves page content, transform it to markdown, and import it (optionally) to a search database to perform semantic (sentences) searches.

[ Parent | on HN ]