As a content consumer, I'm also hoping to be part of the ecosystem. I already use Patreon a lot as "AdBlock absolution", but it doesn't fix the market dynamics. Ad-driven platforms continue to worsen, because they prefer to sell impressions to advertisers than a good product to consumers.
It should be front and center, communicated clearly, and easy to understand. If I was an investor, the lack of clarity after reading a few paragraphs would concern me.
So you use a public resource and presumably like the upside that comes with sharing on it, but you want to limit uses now that someone has found one that you don't like, so you're fine with degrading that public resource?
You could always share things privately or behind a paywall if you don't want them available publicly. But people seem to want to have their cake and eat it too.
I get why a hosting provider would want to limit crawlers to save bandwidth. The "creator" angle is just greed.
Can you say more? what is the "public resource" I'm using as a content publisher? In that role I see myself more as the provider of a public resource (my content), not a user.
And, the idea being proposed is a version of what you're talking about (paywall). Not allowing crawlers to scrap your site unless they compensate you is a form of paywall, just for crawlers...
...But, I do agree with one aspect you're saying about greed, Cloudflare is probably being self serving in this. An open standard seems like it'd be a better solution than having one company control content payment.
Imagine trying to create and distribute a movie without the backing of your local distributor/broadcasting cartel. Only instead of gating the movie theatre, Cloudflare is the single access to all things web. Also, based in the US, so good luck producing content their government doesn’t like.
I have fully read Cloudflare's proposal, as I have RSL, and L402.
In its current state, RSL is not a full solution, but can be. The spec does not cover payments. Its basically just a new token type, which crawlers would register for to crawl your content, and predefined responses for more detailed states. But nothing around payments, yet.
Cloudflare's solution is basically the same as L402. This is very sad for me, because their attempt is basically one of locking down the ecosystem in order to create a monopoly.
If they were really about creating a fair internet where "pay per crawl" is possible (it already was with L402), they would openly support L402, implement it for their gateway, and try to push the industry towards it. But no, they just want the biggest piece of the pie; a pie that doesn't even exist yet.
https://iabtechlab.com/announcing-content-monetization-proto...
Maybe people have agendas, or maybe they have different perspectives that should be reconciled?
It always bothers me a little when people describe themselves as a content publisher or content creator. Why content? Why not say you are a short film maker or writer or musician or videographer or journalist or painter or photographer or ...?
In the past, your work has been merely content to the people who are looking for bits -- any bits -- to place between and around the stuff they really care about: ads. These days I suppose your work may be seen by some as grist for the AI mill.
By calling your work content you are acceding to the idea that your work is only valuable in other contexts. It isn't! I bet it's something you made with care and thought.
Or maybe I misread and you really are a content publisher. You're the person running the CMS who is looking for other people's work to monetize. If so, nevermind.
This is also being attempted by RSL with their “Crawler Authentication Protocol” (https://rslstandard.org/guide/web-crawlers) for demanding proof of licensing from scrapers and RSL Collective (https://rslcollective.org) for providing the licensing itself. The missing piece there is the ability to detect scrapers with high accuracy without punishing regular browsing humans.