Proof-of-work to protect lore.kernel.org and git.kernel.org against AI crawlers

Re:Proof-of-work to protect lore.kernel.org and git.kernel.o... by xxprogamerxy on Wednesday April 02, 2025 @07:48PM (#43563175)

I'm a bit skeptical if this will do the trick. These PoW challenges can be parallelized across different websites and may not be as off-putting as intended. Here some quick back-of-the-napkin math:

DeepMind's MassiveText dataset was sourced from ~2.35B documents. A difficulty of 4 leading zeros requires an expected 16^4 SHA-256 hashes per site. Benchmarks [1] show an H100 at ~12k MH/s, meaning it would take just ~3.5 hours to solve for all 2.35B pages.

[1] https://gist.github.com/Chick3nman/e1417339accfbb0b040bcd0a0...

[ Parent | on HN ]

Re:Proof-of-work to protect lore.kernel.org and git.kernel.o... by xena on Wednesday April 02, 2025 @08:01PM (#43563261)

SHA-256 is a hack to buy time. This will be replaced with something better. It would be faster for me to replace it if I didn't have to do my dayjob: https://patreon.com/cadey

[ Parent | on HN ]

Re:Proof-of-work to protect lore.kernel.org and git.kernel.o... by xxprogamerxy on Wednesday April 02, 2025 @08:16PM (#43563355)

Not meant as a criticism of the project in general. I appreciate people working on this.

I'm curious, what other approaches are you currently considering? In my mind, all roads lead to rate-limiting identifiers with privacy through zk-proofs.

[ Parent | on HN ]

Re:Proof-of-work to protect lore.kernel.org and git.kernel.o... by xena on Wednesday April 02, 2025 @08:18PM (#43563366)

I'm looking at using equi-x, but failing that I may unironically do protein folding.

[ Parent | on HN ]