I did both, but my index is tiny.

I started with a metasearch, since that's (mostly) all Kagi is. I send prepaid-API calls to Brave, Mojeek, Marginalia & Yep for every query, and get about 200 results back on every search. The cost is far less than I paid Kagi. My first version was a single PHP file written by Claude Sonnet 3.6 (!), any modern AI would one-shot something dramatically better. Obviously you can iterate & improve it over time yourself. Grouping the results by domain and ranking by how many search engines return the same page/domain will get you ~60% of the ranking quality.

Over the years I added more ranking signals and had Claude build a ranking optimizer. I provide "golden queries" ranked as I would prefer, and the optimizer runs experiments to optimize the weights for each ranking signal. It isn't perfect, especially since I only use on-page signals. But it has been good enough that I don't miss Google.

My own index is just a couple of SQLite databases. It is ridiculous how far you can get with SQLite, FTS Full Text Search & its built-in BM25. Each webpage takes up 10KB of index storage on average, once trimmed down to just the indexable text. I never clear my Firefox history, so I went through my history cache to find which sites I actually visit, then made a command line indexer to run on my laptop and index them in the mornings. Most sites offer a sitemap or RSS file, so you don't need to worry about crawling the site. My tiny index doesn't answer every query, and that's what the metasearch is for. But surprisingly often, that tiny subset of the internet does have the answer I need, and it saves me a paid search API fee every time it does.

My system is held together with duct-tape and hope, so for most people it's easier just to use Kagi (or Uruky.com). If you do want to build it out, the blog at marginalia.nu has far more useful technical details than anything I can offer. But if you do go down the rabbit hole, I think you'll start questioning how a giant enterprise like Google can manage to screw up search so much.

Do you worry that your methodologies will get you stuck in a local maxima (or minima?). By using combined output of metasearch and perhaps missing overlooked gems?

I treat my Firefox History the same way though the way I use it is a bit different right now. I would like to harness it like yours, but first I think I need to pipe it into ArchiveBox or something