Turn entire websites into LLM-ready data

Re:Turn entire websites into LLM-ready data by jshchnz on Saturday April 13, 2024 @07:37PM (#40027222)

This is cool, seems like a nice way to easily add context for ChatGPT at the least

[ Parent | on HN ]

Re:Turn entire websites into LLM-ready data by cpeffer on Saturday April 13, 2024 @07:44PM (#40027261)

* Creator here - Thats the goal!

[ Parent | on HN ]

Re:Turn entire websites into LLM-ready data by donohoe on Saturday April 13, 2024 @08:16PM (#40027497)

And you honor or ignore robots.txt?

[ Parent | on HN ]

Re:Turn entire websites into LLM-ready data by cpeffer on Saturday April 13, 2024 @08:27PM (#40027557)

It wasn't in our initial version (we didn't plan on launching today), but we are pushing an update to do so now.

[ Parent | on HN ]