There's a third option: allow bots and give them access as efficiently as possible.

As Reddit's API used to.

Don't want to? Tough, they'll do it the hard way and put more load on your servers and your countermeasures will drive away more humans than bots.

Of course our current bot plague doesn't understand this and seems to be optimizing for maximum server load, which is still bad.

Or just block all bots and give them a big F U :)

Why shouldn't bots (who work on behalf of users after all) not get access to information?

I just found something via LLMs and bought it from a website that I would have never found with traditional search engines (because I didn't).

this is bad faith argument. You are well aware that the majority of bots are used for negative purposes, at best, those purposes are neutral.

The situation you describe is certainly legitimate and useful, but is not the norm. Separating legitimate bots from the rest is currently not possible reliably.

Am I a bot? How do you know?

I reject your false dichotomy.

You reject the law of the excluded middle - you reject that either I am a bot or I am not a bot?

This is HK. We here are all bots at heart.

> There's a third option: allow bots and give them access as efficiently as possible.

Why?

To make your data more accessible AND reduce server load. More and more people are using AI, so there will be more and more bots as the trend seems to be accessing data with your AI, and maybe, only sometimes, actually reviewing what it found. I'm afraid that if you don't at least start to consider how we might accommodate the bots, you'll be walling off more and more real users going forward.

what does the site get out of ai users? it doesn't seem like they have a strong incentive to let you not actually visit the site

I'm with ya, but I'm not sure what the solution is. Just pointing out a reality the internet's going to have to cope with.

My cynical take is we'll just start putting ads into the text that targets agents.

A wide portion of the Internet is not ad supported. Consider a user searching for product support: the company benefits from the user self-solving the issue without having a ticket filed.

This, may, however be to the detrimental of discussion platforms, but if that drives people to self hosted discussion platforms instead of relying on reddit, then it is a net positive.

Discourse's most recent blog delves into this and is an interesting read, I will submit that I think.

I'll look for that - I'm thinking about small forums and community sites a lot lately with ai stuff.

> To make your data more accessible AND reduce server load.

Wait, how do you reduce server load by allowing traffic from bots?

> More and more people are using AI, (...)

That's specious reasoning. Sites provide data for their users to consume, and in some cases their business model relies on real people consuming their content.

If you replace "bots" with "scrapers", which is a specific kind of bot, what conclusions do you draw?

> I'm afraid that if you don't at least start to consider how we might accommodate the bots, (...)

Wait, you think this problem is new? What do you think companies do with crawlers and scrapers?

I just told you why.

> I just told you why.

No, not really. Is your opinion too hard to defend?

All repositories on git.zajc.tel can be git clone'd from their URL. The clone URL is also listed on every repository's summary page. Unlike the HTML pages, the clone endpoints are not ratelimited, have no bot check, and give you the entire repository with all history without having to crawl billions of pages.

Guess which option the bots pick.

yep, current bits are extremely stupid or intentionally malicious - never attribute to stupidity that which can be adequately explained by malice

Also never say never :>

Can you please tell whoever it is at Mozilla to stop working on that.

This feedback isn't directed at any one person but that proposal impressively suffers from the simultaneous problems of Being Way Too Fucking Complicated and Proposing to Make the Web Better but Actually Making it Way Worse (for Everyone Except Entrenched Megacorps). It's the exact same problem that Google's "anonymous ad attribution" attestation swarms suffered, and luckily that never came to pass.

From looking at the pictures in that article, it seems like in order for me to be able to browse "protected" sites (aka, most of the corporate-owned web, once they are aware that this exists and turn it on because "bots bad") my browser is going to have to negotiate for some scant amount of "allowed access" tokens or other such thing, through some complicated protocol that halts any chance of us realizing a future for the web that looks any much different from "locked down approved browsers visiting approved sites in an approved way."

The further we allow the Web to stray from cURL http://example.com Just Working, the worse off we all are. So to reiterate, this proposal looks insane and not at all aligned with Mozilla's mission...

> The further we allow the Web to stray from cURL http://example.com Just Working, the worse off we all are.

The problem is we don't hold bad actors accountable. That's how we ended up losing open SMTP decades ago.

Unfortunately my pay grade isn't high enough to tell people what to work on, haha. That said, I am fairly sure that whoever is working on it would also prefer `cURL http://example.com` to Just Work.

But keep in mind that not doing anything isn't going to stop the rise of CAPTCHAs. If you've got specific alternative approaches in mind, especially ones that keep in mind the trade-offs mentioned in that blog post, then I'm sure it's possible to reach out to the author.