News for hackers. Stuff that matters.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nostrademons on Friday September 09, 2022 @05:22PM (#32785079)
Note that if you search [purpose of life], it does not say it has 2 billion results anywhere on the first page. My team removed the blue bar containing that text way back in 2010. You have to hit "Next" or otherwise visit page 2 to get it.

And I'd bet the reason why it's still there (I left Search in 2014) is because < 0.1% of users ever hit the next page. Everybody else just refines their query to a different search. It's a holdover from when search engines were bad (i.e. around 1998) and you had to go through 10 pages of results to get the one you were looking for. As a result, Google expends approximately zero engineering effort on pages 2-20 of the results - I know that in the 4 visual redesigns I worked on, we didn't touch them once. It wouldn't surprise me if the response to flack on this is to just get rid of all pages other than the first one - it avoids the issue entirely and wouldn't affect 99.9% of users.

The technical reason for this behavior, as others have remarked below, is pagination. Ranking across the full result set is a very complex calculation, and it can depend on some factors that are basically random (eg. timeouts and failures in backend servers). It'd make pagination basically useless if the same results you already went through show up on a later page because the ranking is different. This requires that the full result set be cached. You can cache 400-1000 results for each of the queries that the 0.1% of users who actually hit "Next" care about, but you'd have a big issue caching 2 billion results for each of those queries.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nomel on Friday September 09, 2022 @05:28PM (#32785138)
> Note that if you search [purpose of life], it does not say it has 2 billion results anywhere on the first page.

This is false for other search terms. Search for "meaning of life". At the top left of the first result page it says "About 10,620,000,000 results (0.56 seconds)".

Screenshot: https://imgur.com/a/SQq7Snl

Logged into Google, Safari 15.6.1.

edit:

Here's a good one:

"starfish buttercup pickle mouse"

Page 1 of 2 says

> About 589,000 results (0.62 seconds)

Page 2 of 2 says

> Page 2 of about 22 results (0.36 seconds)

Oops! Must have been one of those common "multiply by 26,772" errors.

At the bottom of the page, it says "In order to show you the most relevant results, we have omitted some entries very similar to the 22 already displayed." with an option to show all results.

With all results shown, the total result count goes to 59, so only off by 9,983x! But who's counting? Definitely not Google.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by utopcell on Friday September 09, 2022 @11:30PM (#32787498)
I don't doubt that's what you see.

What I get for: "starfish buttercup pickle mouse"

is exactly one result [About 1 results (0.32 seconds)], which is your YC post.

Removing the quotes gives me [About 34 results (0.63 seconds)].

The difference is that I have setup Google to show 100 results per page by default. I believe that this allows the (very approximate) results estimation algorithm to be more accurate. When I switch to 10 results, I get [About 591,000 results (0.53 seconds)]. Feel free to verify that this is the case for you as well.

There is no foul play here. I work at search serving, I've seen that algorithm. My personal opinion is that nobody cared to improve it because almost nobody (modulo YC crowd) cares.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by antonvs on Saturday September 10, 2022 @01:23AM (#32788038)
> There is no foul play here. I work at search serving, I've seen that algorithm. My personal opinion is that nobody cared to improve it because almost nobody (modulo YC crowd) cares.

In other words, the product is shitty because no-one at the company really cares about it - because of course, that “product” is just a vehicle for a big ad company. On top of that, there’s no meaningful competition. Hopefully we’ll get some DoJ action on this at some point.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by utopcell on Saturday September 10, 2022 @02:56AM (#32788480)
By all means, feel free to use any other search engine that you feel is better than Google. Or better yet, wow us with the better engine that you can build.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by antonvs on Saturday September 10, 2022 @03:27AM (#32788636)
It’s almost as if you didn’t even make it to the second sentence of my comment. Maybe take another stab at it? You’re really going to enjoy the third sentence.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nostrademons on Saturday September 10, 2022 @08:29AM (#32790025)
FWIW my default browser is Brave, and Brave's default search engine is Brave Search. I almost switched back to Safari because the Brave Search results were so much worse than Google. Eventually found that you can set the default search engine in Brave - there's Brave, Google, DDG, Qwant, Bing, StartPage, and Ecosia available.

I'd highly recommend this exercise for everyone who complains that there's no competition for Google, because a.) you're actually trying the competition and b.) you might realize how much Google does for you, and how far ahead they are against the competition. Who knows, you might even find a better search engine, and then next time you can come make recommendations for other HNers.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by throwaway894345 on Saturday September 10, 2022 @03:14PM (#32793720)
A lesser man wouldn’t snark so hard while pointing people to an obviously absurd claim (no competition for Google search). Clearly we are in the presence of greatness!
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by Amezarak on Saturday September 10, 2022 @06:45PM (#32795164)
Google used to be much better. If I could use the old Google, I would. I really do want content beyond the top 500 websites, and sometimes it was on page two. Now it’s not there at all.

Most daily usage now I use Bing for the rewards points, and it’s approximately as good as current Google. Never have I failed to find something with Bing I then found with Google, despite it being a regular occurrence.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by Razengan on Saturday September 10, 2022 @09:42PM (#32796115)
Wow, I had strong suspicions Google was dicks (because of too much anti-user BS in their products) but here is proof!
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nomel on Sunday September 11, 2022 @07:55PM (#32804396)
> Feel free to verify this is the case for you.

Yes, if only one page is shown, it’s accurate. I picked that (unquoted, sorry) phrase because it resulted in two results pages for me, showing a worse case scenario. With your settings, it would require a term with > 100 results to see the fiction.

I actually do assume it’s intentional, because

1. It’s predominant, at the top left of the page

2. I assume your UI team is competent and cares about every pixel shown on the screen, especially predominantly placed ones

3. it’s great marketing

4. It’s absurdly incorrect. 10x could be ubderstandable. It’s off by nearly 27,000x error here.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by utopcell on Sunday September 11, 2022 @08:39PM (#32804589)
It was a good query, I'd have done something similar to replicate.

I can guarantee you that it's not intentional. It was definitely easier to estimate in the past, when a single query was fired on a single corpus for a given user input, but modern web search is vastly more complicated than this.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by ImprovedSilence on Friday September 09, 2022 @08:51PM (#32786611)
I shit you not, I searched purpose of life, got no counter. Searched meaning of life, got a counter: https://imgur.com/a/SfzILul

Both typed into the "url bar", and not search box on goolge.com. The only difference as far as I can tell is I copy/pasted purpose of life, and typed out meaning of life... macos & chrome.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by jackstraw14 on Friday September 09, 2022 @09:10PM (#32786700)
Same here, and when I hit page 20 it changed to "Page 20 of about 198 results (1.22 seconds)"
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by usefulcat on Friday September 09, 2022 @10:07PM (#32787014)
Similar for me.. 8-10 billion results, then on page 21, "Page 21 of about 205 results" and no more pages after 21.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by hypertele-Xii on Saturday September 10, 2022 @05:58AM (#32789219)
But if I click the link to repeat the query without truncating the results do continue beyond page 21.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nostrademons on Friday September 09, 2022 @05:33PM (#32785193)
Must've been brought back after I left, I have no idea what the rationale was.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by citizenpaul on Friday September 09, 2022 @05:49PM (#32785325)
Does that mean you don't use google search? Its been back a long time.

What do you prefer for search if not?

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by frozencell on Friday September 09, 2022 @07:24PM (#32786087)
Google employees don't know what's going about top-level policies unlike a few government affiliated managers like Dean or Page (source: google images). It's possible that Google employees don't see the same content as outsiders. Literally CCP vibes, or Plato's Cavern.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by utopcell on Friday September 09, 2022 @11:35PM (#32787548)
Every Google engineer can see the code that performs this calculation. Google engineers have demonstrated that we are very vocal when we smell foul play.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by frozencell on Thursday September 15, 2022 @11:45AM (#32853803)
The government doesn't have backdoors in Google source codes? or Google engineers are patriotic?
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by throwawaylinux on Friday September 09, 2022 @08:21PM (#32786440)
What do you mean by "source: google images"?
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by throwaway894345 on Saturday September 10, 2022 @03:15PM (#32793731)
I’ve used Google for decades and I’ve never paid any attention to this. I wouldn’t be able to tell you if it was there or not without looking.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by saghm on Friday September 09, 2022 @08:09PM (#32786383)
I think a better question is why it was ever there in the first place if the numbers are so extraordinarily inaccurate.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by Kamq on Friday September 09, 2022 @09:04PM (#32786674)
Based on the explanation above, it sounds like the big number is the "real number" and then they throw away everything except the first couple results when they're put in the cache. Presumably the number of results is calculated before this caching and nobody cares enough to fix it.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by cyanydeez on Friday September 09, 2022 @09:28PM (#32786790)
Yeah, so basically you just can't actually get millions of results.

But others have pointed out that those index numbers don't resolve rationally to results.

It seems the index number is a different match than what they choose to show.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by saghm on Friday September 09, 2022 @11:55PM (#32787658)
Sure, I understand where the number comes from, but I have trouble understanding why someone would program that calculation, see that the number is not actually what a user would experience, and say "okay cool let's put that on the page". It seems hard to imagine any possible motivation other than being purposely misleading, in which case the answer the question "why was it put back?" presumably would have the same answer.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nostrademons on Saturday September 10, 2022 @08:34AM (#32790059)
If the number is in the billions you're not going to experience each of those websites anyway - you'd run out of time in your lifetime.

I suspect (it was introduced before I started, and re-introduced after I left) that it's there to remind users of the work that software performs on their behalf, much like how TurboTax says "$X in taxes saved" or Brave says "9 hours saved blocking ads" or your brokerage says "Total profit: $X". Users tend to forget this, and just assume that they're entitled to the world where the product exists.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by Kamq on Saturday September 10, 2022 @06:14PM (#32794981)
That presumably depends on when the caching was put in place.

If the number was in the thousands in the late-90s, you could probably get through all of the sites.

If the caching was put in later, say, the early 2000s when google started getting really big, the number might just keep growing until it eventually looks ridiculous.

I've never worked at google, but I've seen a few systems that broke in weirder ways when nobody bothered to maintain them after they had outlived their usefulness.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by throwawaylinux on Friday September 09, 2022 @08:20PM (#32786431)
You really think a company would do that? Just go on the internet and tell lies?
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by bergenty on Friday September 09, 2022 @06:19PM (#32785576)
Well how do you know there aren’t 10 billion results
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by saiya-jin on Friday September 09, 2022 @05:39PM (#32785245)
Oh boy do I feel special right now, since I hit 2nd page (or even 3rd etc) quite often since main search became quite a spam-infested over-seod crapfest few years ago for some types of search. Some first results are outright dodgy, often outranking ie official sites and I strongly suspect only malware awaits there.

Such a shame for basically one-trick pony who doesnt understand that staying relevant long term means that trick must be and remain a damn good proposition.

Its true I eventually give up and do ie duckduckgo and if that fails I try to refine searches even more (but this rarely works since I already start with it as default)

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by seesaw on Friday September 09, 2022 @07:13PM (#32786005)
I switched to duckduckgo few years back, and most of my searches work. I still have to use g! once in a while. Web search is such an important function, I wish there was a community funded, Wikipedia like entity for search.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by kyleee on Friday September 09, 2022 @07:36PM (#32786160)
we're seeing a small rennaisance of search engines which is promising to me; brave with it's discussions and goggles, kagi, marginalia, million short, and i am sure others it makes me happy to see
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by userbinator on Friday September 09, 2022 @08:36PM (#32786530)
I still remember, around a decade or two ago, going through page after page of results to look for the exact page I wanted, and somewhat often getting that result. Now, if I'm very lucky, I'll find that result; otherwise, I'll probably get a CAPTCHA-hellban instead.

It's very ironic that a search engine, which is supposed to be helpful precisely for the things that are hard to find, seems to actively sabotage your efforts at such deep searching.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by echelon on Friday September 09, 2022 @05:43PM (#32785271)
What does the search org look like internally? Is it connected with the ads or Chrome orgs at any interface? Do the rank and file ICs, EMs, and PMs have issues with how these products interact?

I have so many complaints about your product, it drives me wild.

How many people even look at organic results versus the paid-for ads that display at top?

It should be illegal for anyone to be able to purchase ads for another company's trademark. Apple, Google, and Amazon are all extorting companies by forcing them to buy ads to protect their own brand. (My own brand is being attacked by a competitor in this way, and it's ridiculous!)

The only reason anyone uses these systems at scale is that third parties were available in aggregate early on to provide content. You built your product off of our backs. And now that the power dynamic has shifted, we're cattle to soak for as much revenue as possible.

It also seems like the only reason Google is dominant is bad behavior. Paying for default search engine status. Being the default in all of their other unrelated platforms. Achieving browser monopoly.

I've recently started seeing Chrome ads and billboards everywhere. Google purchased a huge percentage of my city's billboard ad inventory for their "better on Chrome" campaign. It's as if Google knows this is the reason for everything. Where except for Apple devices is Chrome not dominant?

This whole cartel needs a muzzle.

Device companies should not be ad companies.

Ad companies should not be service companies.

Service companies should not be content and production companies, since they can favor their own and price pressure the rest.

We have a world where the top tech conglomerates are all of these and then some. They've cast a wide net and turned the whole world of consumer interaction into a supermarket, where we now have to pay for "shelf space" to interact with customers, pay to protect our brands from unfair sniping, pay to grow, obey asinine rules to build a product that fits their desired shape, integrate with their payments and login stack (so we're even less in a relationship with our customers).

It's a far cry from the open web of the 90's. Really bad for small companies, new startups, and even consumers. We can barely afford to build our products with all the margin that goes to Google, Apple, and the rest.

I wish the rank and file could feel this. :(

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by collaborative on Friday September 09, 2022 @07:14PM (#32786019)
I closed my small shop due to not being able to afford appearing on ads anymore, so I feel your pain. You are being downvoted because most people here benefit from the $$$ big tech makes from ordinary hard working people. And they think complaining about the unfairness of their practices is petty. Talk about lack of empathy
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by johnny35 on Saturday September 10, 2022 @06:21AM (#32789323)
Thanks for this comment. I have personal experience with all of these frustrations and agree very strongly. I can see someone writing off this comment as just another rant but it really isn’t - you’re hitting the nail right on the head over and over.

The way Apple and Google have wedged themselves in as an interface and then gatekeeper to our customers is deeply pernicious. I respect what they have achieved but it’s deeply saddening to wonder if the freedom we had decades ago online was anomalous and not intrinsic to the medium.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by nostrademons on Friday September 09, 2022 @05:49PM (#32785322)
I have no idea, I left Search 8 years ago. I would bet on people responding to incentives, though, because that seems like a universal constant of human behavior.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by effingwewt on Saturday September 10, 2022 @06:44PM (#32795157)
That's a bullshit cop-out if I've ever heard one.

'Don't mind us destroying society, we are just following incentives!'

Swear I've heard that somewhere before though...

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by zarzavat on Saturday September 10, 2022 @01:55AM (#32788190)
> It wouldn't surprise me if the response to flack on this is to just get rid of all pages other than the first one - it avoids the issue entirely and wouldn't affect 99.9% of users.

You are confusing users with queries. A lot more than 0.1% of users use the other result pages - double digit percent certainly. But if Google is doing a good job they don’t do it very often.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by ulfw on Saturday September 10, 2022 @04:14AM (#32788827)
I am sorry but it doesn't change the fact that it is super disenginous to say (I just tried it myself): and then you only display a maximum of 149 results.

"Page 14 of about 184,000,000 results (0.83 seconds)" "Page 15 of about 149 results (0.76 seconds)"

"In order to show you the most relevant results, we have omitted some entries very similar to the 149 already displayed."

Maybe it's time Google spent a bit more time on their search engine, including pages 2-20.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by dmitriid on Friday September 09, 2022 @06:21PM (#32785607)
> It's a holdover from when search engines were bad (i.e. around 1998) and you had to go through 10 pages of results to get the one you were looking for.

Unlike today, right? Where actual search results... represent less than 10% of the page and the rest is irrelevant information and ads in Google search https://grumpy.website/post/0XCmMC-2O

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by debesyla on Friday September 09, 2022 @08:25PM (#32786473)
In your example (link) one of the engines returned way better results: guessed and returned stuff that the query author probably meant.

In my opinion a search engine that can guess and understand my question from a single word is superior to encyclopaedic-bureaucratic return. (After all, in whole lot of times I am searching for something that I don't even understand what am I really looking for.)

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by dmitriid on Friday September 09, 2022 @08:45PM (#32786573)
You can also scroll down to comparison between 2000, 2013 and 2020 here: https://www.washingtonpost.com/technology/2020/10/19/google-...
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by wolpoli on Friday September 09, 2022 @07:25PM (#32786090)
> The technical reason for this behavior, as others have remarked below, is pagination.

> It'd make pagination basically useless if the same results you already went through show up on a later page because the ranking is different.

A bit off topic: I have always wondered why search result is the only feed on the web (apart from HN) that has not adopted infinite scrolling. This is a good explanation.

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by jtc331 on Friday September 09, 2022 @08:33PM (#32786510)
Infinite scroll is a terrible UI experience despite being popular; that’s why not everyone has adopted it.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by wolpoli on Friday September 09, 2022 @11:35PM (#32787544)
For sure, infinite scroll isn't designed for the benefit of the user.
Re:Google pays ‘enormous’ sums to maintain search-engine dom... by belter on Sunday September 11, 2022 @10:42AM (#32800001)
Searching "purpose of life" while logged in shows about 245 results, searching "purpose of life" while logged out shows "Page 10 of about 10.740.000.000 results"...

Extreme profiling Google? :-)

Re:Google pays ‘enormous’ sums to maintain search-engine dom... by JohnFen on Monday September 12, 2022 @01:37PM (#32813872)
> It's a holdover from when search engines were bad (i.e. around 1998) and you had to go through 10 pages of results to get the one you were looking for.

Search engines are still bad.

Having to go through 10 pages of results to find something that matches what I was looking for is one of the reasons I stopped using Google. What I want is almost never on the first few pages of search results.