Re:The Future of AI Relies on a High School Teacher’s Free D...
by wildrhythms on Tuesday April 25, 2023 @07:43AM
(#35699109)
robots.txt generally only addresses permission to crawl; not permission to reproduce the content.
It obeys robots.txt, and the user-agent is documented.