I get Baby Mama as #1 as well, but at least, from the movie synopsis, it doesn't appeat to be an irrelevant result.
Edit: Oops turns out its 11 years old. Better make that the last two decades then. I'm gettin old
I've heard that a key early advantage of YouTube was that uploaded videos appeared immediately, not after a long processing delay. That helped it become popular, and once it was popular, it stayed popular and crushed the competition due to the network effect.
I use https://chrome.google.com/webstore/detail/personal-blocklist... but it should come with pre-compiled lists, like uBlock Origin.
This feels like an application of the 80/20 rule. I might not always find what I need in just that offline index, but the times I do would seriously disrupt google.
Oh, I wish there were a way to rate the results in a opposite way. Maybe it's just me, but sometimes I want even mark some site as 'a hidden gem'. Usually it's something rather niche though, like a bunch of great little articles about anti-aliasing filters design and sampling, full of engineering wisdom from the decades of experience. So I think it would be perfect to have a way to tag it by some topic.
I don't have a CS background so I'm not sure, is this a reasonable solution?
But having a way of making sure that a certain topic or genre of sites is well indexed locally would work. I.e. "my" top 100k. Oh, how awesome would that be.
Laptops generally come in 128GB or 256GB capacities. So we're talking roughly 20-50% of an average laptop's storage just to hold the search index for wikipedia alone.
Meaning it is not feasible to store a useful index of the web today on a laptop unless you want to dedicate that laptop to pretty much exclusively searching wikipedia's english content.
You're not searching that database in milliseconds on average laptop hardware, either, but you probably could get it to be "fast enough" for practical usages if you could somehow solve the harder parts like storage size, freshness, and indexing more than just wikipedia's english content.
Your indexing performance would also sharply decline when subjected to the random access seek times of those 2.5" spinning metal.
>That's an rough approximation of all known human knowledge.
Are you taking the piss?
Instead, I am wondering why there isnt a federated open source search engine. It would be a cloud of nodes, each node spidering and indexing a small hash bucket of urls. With a million such nodes we could have a live updated, distributed search engine to replace Google. We could run millions of queries without paying someone - we'd pay back by serving as a node, just like in BitTorrent. With all the interest in privacy, I wanted to see more discussion of replacing Google with an open, non-censured and private protocol for search.
Not sure if federated by they call themselves decentralised: https://yacy.net/en/index.html
You vastly underestimate index sizes. I'm not saying it wouldn't be realistic to download and index locally but they'll be much larger than 1-5%.
Offline search is the equivalent of the Internet Yellow Pages from back in the day. New, relevant information is being added continuously. The search index 10 minutes ago is different than it is right now.
Relevant discussion: "Wallabag: a self-hostable application for saving web pages" | https://news.ycombinator.com/item?id=14686882 (July 2017)
Is there a way to easily sync search repositories, like Stackoverflow, Wikipedia etc., to your local computer with an automatically built search index?
the problem is this would be completely useless for current events. but querying sites you use regularly, this could be an option. I often end a google search with modifiers like "wikipedia" or "reddit" or "stackoverflow". If I could store indexes to those sites offline that might be useful.
The creator of this offline search could potentially make money by indexing shopping sites like Amazon, eBay, etc with affiliate links.
Your reply seems pointless, yes I could download an entire copy of Wikipedia, compress it and index it - why would I want to? Are you going to do this on every machine you own? What about the updates Wikipedia receives every minute?
You should start a company though to focus on this, maybe you could call it Encarta or something?
This was true when Google was created.
No one had the processing or memory available on their desktop to search an entire index of the "useful" web.
Not anymore.
How large is a "useful" index of the web today? And can it fit on your laptop? The answer is yes.
Can the entire thing be queried fast? The answer is yes.
As an example take the entire stackexchange and wikipedia dumps in their entirety(including images). Compressed it comes to 50-60 GB range. Think about that number. That's an rough approximation of all known human knowledge. It's not growing too fast. It has stabilized. To query the content you need an index.
So how large is an index to a 100 GB file? Generally around 1 GB. Let's say you use covering indexes with lot of meta data and up that to 5GB to support sophisticated queries.
With today's average hardware you can search the entire thing in milliseconds.
So why aren't we building better local search?
Because everyone is conditioned to believe, thanks to Google's success, we need to do it online. Which means baking in the problem of handling millions of queries a second into the Search problem. Guess what? This is not a problem that local search has.
Every time a chimp or a duck needs to build a protein in it's cell it doesn't query the DNA index stored in the cloud. Instead every cell has the index. Every cell has the processing power to query that index in the nanosecond time scale.
The cloud based search story is temporary.
If you want to index every reference to Taylor Swifts ass that every teenager in Norway, Ecuador and Cambodia are making, then yes you need a Google size index. But for useful human knowledge we are getting to the point where we don't need Google scale.
If you don't believe me look at what is possible TODAY in Dash/Zeal docset search for offline developer documentation search or Kiwix or with Mathematica.