News for hackers. Stuff that matters.
Re:A look at search engines with their own indexes (2021) by marginalia_nu on Tuesday June 21, 2022 @03:51PM (#31828335)
The problem with this is that you get significant benefits from a larger crawling corpus, even if you index just a small portion of it, the rest will inform that portion.

That's a real problem that is ultimately hard to get around. Like my index is fairly small as it is, but even so it requires more hardware than say a student could afford. Like it's not enough that I'm dependent on external funding, but it's still not something you slap together for fun and then get bored of.

There are no comments on this comment yet.