Re:The Evolution of Marginalia's Crawling
by marginalia_nu on Tuesday August 23, 2022 @01:38PM
(#32568324)
Yeah, probably. Could also be the answer isn't more complicated than to offer up the ability to control the search a bit more. Especially with forums it's easy to figure out the post time since there's only some half-dozen forum softwares. Maybe just add the option to filter by year or something would go a long way.
I do think offering different filters like that is probably a good option to having the search engine try to mind-read what you want based on spying on your historical queries and machine learning haruspicy.
Yea, I don't think the idea has any real value. It kind of falls in between a bookmarking tool and a search engine. But forming search engines from trees of other search engines is kind of cool, and I have stumbled onto knowledge that I wouldn't have otherwise found. I'm not sure what the balance is between mind reading and offering too many knobs to a user is, though.
I'm working on a federated search tool. I've built a few different sectors (I've called them "realms") that I care about (programming, automotive, racing, fediverse) and found that I need pretty different heuristics depending. Example: car forums circa ~2005 are a treasure trove of valuable information, but 17 year old posts about programming are (in general) less interesting. That informs a lot about how URLs need or do not need to be re-crawled.