News for hackers. Stuff that matters.
Re:Google emphasizes popularity over accuracy by Someone1234 on Tuesday March 31, 2020 @10:06PM (#22744916)
I'm just glad I'm not alone, sometimes I second guess if I am the problem.

Currently just to get basically workable results I'm finding myself putting "every" "keyword" "in" "quotes" (to make sure they're actually in the result pages at all), the site: modifier to restrict it to real sites, and negative modifiers (-"keyword") to remove some SEO results.

Google used to be "magic" in that it knew what you were thinking, and gave you what you wanted instead of what you asked for. These days it is just page after page of auto-generated results, pages that don't contain anything relevant to your query, or just low quality results.

I'm not going to pretend I'm an expert in Google's search, and I'm sure they're meeting some metric or another, but from my perspective things have gone seriously downhill to the point where I am looking elsewhere.

It used to be THE technical search engine de jure. Now it feels like a search engine you have to hack to get it to work well. Not a good place to be.

PS - I have read, in HN comments (so pure rumor) from self-proclaimed ex-Googlers, that Google's internal culture punishes people for improvements/maintenance to existing products, and that promotions come from developing new products/features. If even semi-accurate might go a long way to explaining why Google Search feels neglected aside from changes which seem to exist to improve their button line/promote sister products.

Re:Google emphasizes popularity over accuracy by int_19h on Wednesday April 01, 2020 @12:45AM (#22745780)
The problem isn't just SEO, it's that Google itself aggressively rewrites queries to produce more results (which I suspect they want to do to show more "relevant" ads).

On the most extreme end of this, I've seen four-word queries produce results, in which three of the words were stricken out. More often, it's just one word, but it's usually exactly the one that makes the difference between a very specific query, and a very generic one.

Worse yet is that they try to do synonym substitution, but their algorithm has a ridiculously low bar for that. Like, you might be searching for "FreeBSD", and it will substitute that for "Linux", or even "Ubuntu". Or search for a specific firearm model, and it finds "gun".

Quoting keywords suppresses all of that, but synonyms are actually useful - if it did them accurately...

Re:Google emphasizes popularity over accuracy by KMag on Wednesday April 01, 2020 @08:24AM (#22747889)
I left Google in 2010, so it's just a wild guess, but I suspect a big part of the issue is learn-to-rank is probably being trained on everyone's searches. I think it would probably do much better if they used the presence or absence of search operators as a simple heuristic to separate power user searches from common searches, and trained a separate ranking model on power user searches.

Maybe they're already doing this, but it sure acts like learn-to-rank is always ranking pages as if the query were very sloppy.

It's been a long time, and I certainly never read the code, but I vaguely remember a Google colleague mentioning something (before learn-to-rank) about a back-end query optimizer branch that would intentionally disable much of the query munging if there were any search operators in the query. There was some mention about using cookies / user account information to do the same if the same browser/user had used any search operators in the past N days, but I'm not sure if that was implemented or just being floated as a useful optimization.

Re:Google emphasizes popularity over accuracy by lowdose on Wednesday April 01, 2020 @07:17AM (#22747555)
Google image search is also not searching for a duplicate but what object the ml recognized on the picture. For image search I switched to Yandex.
Re:Google emphasizes popularity over accuracy by rathel on Wednesday April 01, 2020 @07:18AM (#22747562)
I wonder if this synonym substitution was their use case leading to invention of word2vec.
Re:Google emphasizes popularity over accuracy by redwall_hp on Tuesday March 31, 2020 @11:20PM (#22745307)
I think it has to do with shifting expectations. All of us who use the Web seriously, and have been for years, want a full text search engine. The average user wants what Ask Jeeves promised to be: something that takes vaguely question-shaped queries and spits back a fast answer. Or a glorified URL bar to outsource memory and effort.
Re:Google emphasizes popularity over accuracy by not2b on Tuesday March 31, 2020 @11:55PM (#22745490)
No, you don't want a full text search engine. If you think you do, you don't remember the pre-Google world. It was impossible to use the older search engines to find a reasonable explanation of a common topic, because to Alta Vista and other search engines of that era, every page that contained a given term was considered equal to every other page, and it would give you all of them in a random order. You could add lots of AND and OR to try to exclude what you didn't want, and this might cut you down to 40 or 50 pages to go through to maybe find what you want.

But when Google first came out, it was a shock. You could just search for something like "Linux", and the most authoritative sites all showed up on the first page.

Re:Google emphasizes popularity over accuracy by userbinator on Wednesday April 01, 2020 @12:00AM (#22745519)
and this might cut you down to 40 or 50 pages to go through to maybe find what you want.

At least those search engines gave you that many results to go through... now Google gives you less than that, full of spam (despite the index probably containing far more), and you'll be in CAPTCHA hellban if you try harder to get to the rest.

Re:Google emphasizes popularity over accuracy by miracle2k on Wednesday April 01, 2020 @05:40AM (#22747059)
A full text search with a good ranking is still a full text search. The point here is that Google used to do the job just fine, but no longer is.
Re:Google emphasizes popularity over accuracy by kevin_thibedeau on Wednesday April 01, 2020 @03:52AM (#22746637)
AltaVista's killer feature was the NEAR operator.
Re:Google emphasizes popularity over accuracy by jkaptur on Wednesday April 01, 2020 @12:00AM (#22745516)
Yeah, I'm sort of surprised that there isn't a semi-popular "web grep" tool for people who would rather use regex, some understandable ranking algorithm with knobs to tweak, etc.

Of course, you'd have to read a manual to use it and it would have a ton of spam, but some people just want lower-level control - they still sell stick-shift cars.

Re:Google emphasizes popularity over accuracy by zo1 on Wednesday April 01, 2020 @02:59AM (#22746384)
Not just that, but the sheer scale of such an index. The size of the web now just makes anything small next to impossible without a lot of funding. And none of the existing search engines will probably allow you programmatic/data access to their index without a metric ton of cash
Re:Google emphasizes popularity over accuracy by artificial on Wednesday April 01, 2020 @03:23AM (#22746503)
How is it that spiders/bots are able to "index" copyrighted content? Is it just one of those things where the ends justify the means or a holdover/tradition or some such?
Re:Google emphasizes popularity over accuracy by kd5bjo on Wednesday April 01, 2020 @03:47AM (#22746603)
It’s some combination of fair use and raw data not being copyrightable. My understanding is that only the creative expression that’s copyrighted, and not the actual words. So, if you distill out all of the creativity into something that’s purely information about the work, you’re probably fine copyright-wise.

There’s a long tradition of compiling and publishing concordances, which are just indices of every place each word appears in the original text. They’re generally not useful without access to the original, so noboy seems to mind them very much. Google’s index is just a modern form of the same thing.

Re:Google emphasizes popularity over accuracy by sefrost on Wednesday April 01, 2020 @03:05AM (#22746413)
I wonder how many pages would be worth indexing for such a tool though.
Re:Google emphasizes popularity over accuracy by zo1 on Wednesday April 01, 2020 @08:22AM (#22747883)
Probably a tiny subset. But the problem is finding that small subset!
Re:Google emphasizes popularity over accuracy by nradov on Tuesday March 31, 2020 @10:29PM (#22745011)
Basically now we're back to forcing Google to work the way AltaVista did in 1997.
Re:Google emphasizes popularity over accuracy by neotokio on Wednesday April 01, 2020 @08:29AM (#22747918)
I think there are few things to consider here,

1) Google CANNOT provide you with technical search, because choice of index/query filters is always limited (ie. Do I prefer exact matches over multiple matches?)

2) Google has shareholder & public responsibility. It means that service is adjusted (and it's 'algorithms') towards biggest type of queries performed.

All of this is a constant battle between precision and recall for given query. Adding to complexity, Google needs to account for

* Extraordinary amount of users using their search

* Extraordinary amount of data on webpages

* Importance of authority

In smaller search engines (ie. shop full-text search) you usually adjust towards one use case. This in itself is already hard.

Google does that for all possible use cases, for all possible queries while still fighting same precision/recall battle.

To be clear. I think google is terrible, but I also think that there is no other option for them at this point.

All of this became clear for me the moment I've got interested in build search and relevance engines.

Re:Google emphasizes popularity over accuracy by MaxBarraclough on Wednesday April 01, 2020 @06:48AM (#22747403)
Presumably google are in an arms race with the spammers, like all search engines are.

Is there a superior alternative to Google?

Re:Google emphasizes popularity over accuracy by wizzwizz4 on Wednesday April 01, 2020 @09:27AM (#22748264)
DuckDuckGo does something fancy with smooshing together Google and Microsoft databases with their own to create a half-decent search engine.

Cliqz has hardly anything indexed at the moment, but it actually gets relevant results from those. (e.g. "zoom privacy" brings up the Zoom privacy policy first, then three news headlines from the last 12 hours, then a news article from yesterday, then an IT@Cornell guide for making Zoom meetings private, then some more news articles, some stuff about HIPPA…) I really like it, even if it isn't great for programming at the moment.

[DuckDuckGo]: https://duckduckgo.com/ [Cliqz]: https://beta.cliqz.com/