Re:Nepenthes is a tarpit to catch AI web crawlers
by jsheard on Thursday January 16, 2025 @09:48AM
(#42725898)
Real search engines respect robots.txt so you could just tell them not to enter Markov Chain Hell.
[ Parent
| on HN ]
Re:Nepenthes is a tarpit to catch AI web crawlers
by throwaway744678 on Thursday January 16, 2025 @10:12AM
(#42726318)
I suspect AI crawler would also (quickly learn to) respect it also?