News for hackers. Stuff that matters.
Re:Serve Markdown to AI Agents with Accept Headers by k1m on Wednesday August 26, 2026 @09:45PM (#49458412)
Agree. I think many people forget that not long ago, HTML markup on many sites was a lot richer than it is today. Making it trivial to produce a good trimmed down markdown version.

The reason it may be more difficult today is because we've lost a lot of that. Some of it because of modern JS frameworks, but some also because publishers simply don't want to make it easy for the useful stuff to be scraped and extracted easily.

I'm not convinced that's changing because of AI agents (it's getting worse in many ways with anti-agent rules). Maybe improving for documentation pages intended for agents. But if it is changing, I think it'd be far easier to improve the HTML and let the agent take care of the rest.

There are no comments on this comment yet.