I'd also like to mention that I'll be using [0] as a template for how to describe to the public my own process of researching and my though process for picking and choosing a particular piece of tech/data structure/solution.
Just a splendid read.
[0] https://boyter.org/posts/abusing-aws-to-make-a-search-engine...
You can follow me on twitter, or use my blogs RSS, or email if you like. You can get all details on my profile :)
Distribution seems like the exact opposite of being reliable? How would the network defend against bad actors?
However you would probably want to have them return ranking information allowing for re-ranking on the caller including its own system and others. Perhaps returning this could be optional?
Or even have the federated searches share portions of the index when peering others. This would allow one system to determine if the ranks appear in line with expectation. Allowing one to trust but verify.
Ah, “it would not”, sounds absolutely great.
I have often wondered if perhaps a distributed search engine is the real answer. Yacy was interesting but the results terrible. But perhaps it’s possible to use activitypub protocol to build a distributed search that has different implementations on the backend. It’s something I keep toying with in my spare time, and with what I did with bonzamate we’re the search is done in lambda functions on aws possible to roll out in a way that’s cheap for everyone to build out. The ability to federate between those providing good results, even at query time has the ability to be very powerful if implemented correctly.