I know enough to know I don't know much :) But there's plenty of documentation out there.
My impression is that if you're not dealing with people trying to game the algorithm then it's all a lot simpler.
As to data sources - with a constrained set of sites the creation of custom adapters should be do-able I'd have thought.
Not that I have any plans. Maybe I should though :)
Edit: come to think of it, it's practically an internet law that stuff you think of already exists. So maybe I should just post an Ask HN for the link to the search engine in question...
Re:Ask HN: Is Google Search Dying?
by kingsuper20 on Wednesday May 05, 2021 @11:26AM
(#27051116)
An interesting meta-problem comes with success of a curated search engine. The sites it actually supports would themselves have to be curated as otherwise the camel's nose gets back into the tent.
I know basically nothing of the mechanics of search, although I expect that a boutique search engine would have trouble accessing the guts of third party sites as they don't want to get hammered by the world.
I did just think of an interesting specialty engine. If you had fast access to the full library on libgen or sci-hub, a person could produce a pretty interesting set of abilities.