News for hackers. Stuff that matters.
Re:Three areas where Google Search lags behind competitors: ... by belter on Wednesday April 13, 2022 @05:17PM (#31020403)
Still broken after 7 months.

"When did Neil Armstrong set foot on Mars?"

https://news.ycombinator.com/item?id=28224730

Re:Three areas where Google Search lags behind competitors: ... by tshaddox on Wednesday April 13, 2022 @05:37PM (#31020613)
"Broken" is a stretch. It gives the only result that could be reasonably expected by anyone making that query in good faith. If you're genuinely expecting a result like "never," then I wonder, would you also expect a result of "never" for the query "When did Neal Armstrong set foot on the Moon?" (notice the misspelling of his name)
Re:Three areas where Google Search lags behind competitors: ... by lordnacho on Wednesday April 13, 2022 @05:48PM (#31020715)
For instance I saw that many Russian generals had been killed during the war. I wondered how many Ukrainian generals had been killed over the same period.

Still can't figure it out. Lots and lots of articles about dead Russian generals.

There's got to be some way where the search engine doesn't second guess you. You can never find the answer to something adjacent to a popular question with the current state of things.

Re:Three areas where Google Search lags behind competitors: ... by PeterisP on Wednesday April 13, 2022 @09:05PM (#31022281)
Ukraine has repeatedly asserted that the HQs of all their major units are intact and as far as I know there have been no announcements of any Ukrainian generals killed and as far as I see even Russian media have not contested that. With some searching on the Russian internet, I find some assertions in Russian social media from early March that Lt Gen Юрій Содоль commanding the Ukraine Marines was killed, but not anything conclusive. So the number seems to be 0, or perhaps 1. And IMHO that isn't surprising, it's quite normal to fight major wars without many generals KIA, it is very surprising that Russia lost so many of them.

Even on colonel level the losses seem to be small - I think that this https://www.republicworld.com/world-news/russia-ukraine-cris... this was the first Ukrainian ground force colonel KIA that I recall, there was at least one air force colonel before that.

Re:Three areas where Google Search lags behind competitors: ... by Spooky23 on Wednesday April 13, 2022 @10:34PM (#31022858)
Ukraine has better signals security so we don’t know.

The other thing is that Russia is operating WW2 style. It’s a top down system where the lower level people have no autonomy. American colonels and generals die in helicopter shoot downs and accidents. Russian generals get assassinated on the front screaming at soldiers to move trucks, etc.

Re:Three areas where Google Search lags behind competitors: ... by ______-_-______ on Wednesday April 13, 2022 @06:52PM (#31021294)
From google's point of view, it's not second guessing you. It just associates words it decides are similar. You'd probably accept results for "dead generals", "killed generals", or "killed commanders". To google, dead = killed, generals = commanders, Russian = Ukranian, and moon = mars. It will expand the meanings of words until it has a page full of results and ads to show you.
Re:Three areas where Google Search lags behind competitors: ... by dave5104 on Wednesday April 13, 2022 @05:52PM (#31020759)
I'd consider it broken. In many other cases, if I make a typo in my search, Google will (rather prominently, IMO) at the top of the search display something like:

> Showing results for "When did Neil Armstrong set foot on the Moon?"

> Show instead for "When did Neil Armwrong set foot on the Moon?"

However, it seems to only work for actual spelling errors (e.g. if I typed Armstrong incorrectly) vs. context errors.

So to answer your question, I'd expect Google calling out its interpretation, but not necessarily a "never".

Re:Three areas where Google Search lags behind competitors: ... by makeitdouble on Wednesday April 13, 2022 @06:08PM (#31020915)
What is “good faith” in that case though ? Some people will really not know (think kids for instance, but not only)

The interesting part to me is that Google will show redirected queries from typos (“Did you mean “When did *Neil* Armstrong set foot on Mars” ?”) but not disclose the term it completely ignored.

I makes it look like it parsed and validated all the search terms before coming up with the prominently displayed date, which makes it worse.

On your point, it’s broken because it shouldn’t display “Never”, and instead skip the date widget and show that the page results are for the moon, like it does for typos and other kind of corrected search terms.

Or alternatively _always_ show which part of the query the results are based on. Failure to be transparent creates the problem.

Re:Three areas where Google Search lags behind competitors: ... by tshaddox on Wednesday April 13, 2022 @06:25PM (#31021053)
> Some people will really not know (think kids for instance, but not only)

It seems very improbable that a kid would ask that question without having any preconceived notion whatsoever of a person with that name famously stepping foot on an extraterrestrial body. It's overwhelmingly likely that they have the famous 1969 event in mind, and that's why Google's response is appropriate.

Of course I'm not disputing that it wouldn't be even better for the Google response to explicitly correct the mistake in the query. I'm only disputing the fact that it's "broken." If you were asking this question of a human as some sort of test (but the human didn't know they were being tested), it would clearly be a "gotcha," and you could likely "fool" highly educated people who know full well that it was the Moon and not Mars.

Re:Three areas where Google Search lags behind competitors: ... by makeitdouble on Wednesday April 13, 2022 @08:50PM (#31022158)
> It seems very improbable

This is I think the critical point.

It’s one of the core assumption that is correct for us to have in our day to day interactions with other real people, and we also are able to adjust the probability levels looking at the person or the context, and follow up depending on the reaction to our answer.

None of that applies to Google Search [0], and we are fed with “most probable” results without qualifiers, little to no sensible adjustments, and very little control or opt out options.

Billions of people use Search every day, at these scales what is “improbable” actually happens millions of times, and I feel too many people are willing to throw the odd ones out under the bus, even if the current situation isn’t perfect either for the 90 part of the 10/90 split.

[0] search personalization based on logged in profile could be used, but in practice they only apply that to very crude adjustments like country, language and frequent searches.

Re:Three areas where Google Search lags behind competitors: ... by tshaddox on Wednesday April 13, 2022 @11:35PM (#31023166)
There is something to be said about how failure cases are handled even for systems that are almost always (say, 99.9%) successful. This is particularly true for systems where failure cases cause harm to people. A common example might be assumptions about people's names that are implemented in some form on a government website. Perhaps it is the case that 99.9% of people in this jurisdiction do have a first name and a last name each of which are representable in Unicode with a character count less than 50. But that's still 0.1% of people who now can't get a driver's license, and that's unacceptable.

However, in this particular case, I think the very low probability of someone genuinely asking when Neil Armstrong set foot on Mars, combined with the very low probability of any measurable harm being done to those people by Google's response, makes me conclude that this is reasonable expected behavior and not something I would call "broken."

Re:Three areas where Google Search lags behind competitors: ... by makeitdouble on Thursday April 14, 2022 @01:21AM (#31023661)
You are bringing a very good point.

On the Google side, I think it’s an issue that has more consequences than just the moon landing.

For instance, for me “When is the francis election” (where I would have men “french or france instead of francis”) gives me a big and bold “Mar 13” with smaller below “Anniversary of the election of Pope Francis Observances” and an long anniversary Year/Week/Date table taking 80% of the widget display.

And as with the other examples, there is nothing showing the word approximations that has happened regarding to the original query.

There must thousands of other instances where a search result will come up with a big widget, an answer in big and bold font, except it will be completely wrong and have a direct impact on the user missing a deadline or taking the wrong action.

Of course users are supposed to know better and check the full result, but as you point out, if it’s almost always what they expected, they’ll learn to rely on it and be more complacent.

Re:Three areas where Google Search lags behind competitors: ... by cryptoz on Wednesday April 13, 2022 @06:37PM (#31021190)
The answer Google gives to the question is wrong. End of story. Google aims to be a question-answering machine with the Info box, so giving the wrong answer is a broken state. Not sure you can really reason you way into how it's kinda-understandable-and-gotcha! to be 'not broken'.

Aside: Going to get real confusing when another Neil Armstrong born in the 2010s does set foot on Mars in the 2030s. I wonder if we will still get the first response from Google, 1969, and if you will still consider it to be 'not broken' then too.

Re:Three areas where Google Search lags behind competitors: ... by seoaeu on Wednesday April 13, 2022 @09:33PM (#31022444)
How is this different from answering the query "2+22=?" with "4"? There's a factual question that was asked and the answer provided is objectively wrong. Whether the user meant to ask a different question is irrelevant. Either say "2+2=4" or say "24", but under no circumstances say "4" without noting that you've answered a different question than what was asked
Re:Three areas where Google Search lags behind competitors: ... by dodobirdlord on Wednesday April 13, 2022 @11:33PM (#31023159)
It's different because nobody is actually going to ask this silly question in good faith. It does not matter in the slightest bit to anyone whether Google can produce a correct answer to nonsense questions that nobody is ever going to actually ask with the intention of getting a correct answer. Google also can't produce correct answers to queries written in ancient undeciphered languages. Oh well.

It also seems rather unlikely that someone would search for "2+22", but not obviously any more or less unlikely than "2+2", and building a calculator into your search engine is trivially easy compared to handling natural language queries, so it's not a useful comparison.

> Whether the user meant to ask a different question is irrelevant.

Of course it's relevant. If Google can correctly determine what question the user meant to ask, then obviously Google should provide an answer that provides value to the user instead of trolling the user with nitpicking about spelling or what have you. If you ask Google for the "capitol of India" you'll get New Delhi, even though that is the "capital" of India, and arguably the "capitol" of India is actually the Chandigarh Capitol Complex, so the result is "objectively wrong".

Re:Three areas where Google Search lags behind competitors: ... by tshaddox on Wednesday April 13, 2022 @11:41PM (#31023191)
Google can interpret the intended meaning of arithmetic expressions like that with even higher confidence than this Neil Armstrong query. But the confidence with which Google can interpret the intended meaning of this Neil Armstrong query is well above any reasonable threshold one might propose.

I would argue that it's roughly just as clear that the query intends to refer to the Moon as it is that the query intends to refer to the Canadian Neil Armstrong who was killed in a plane crash in the Antarctic in 1994.

Re:Three areas where Google Search lags behind competitors: ... by wdevanny on Wednesday April 13, 2022 @06:34PM (#31021141)
At the very least I would expect a "Did you mean..." message at the top. In fact, it correctly does this for the "Neal" query. The lack of one indicates a failure on google's part to understand the answer it gives.

Yes, people probably don't make this exact query in good faith very often, but it raises significant concerns for other queries where google tries to supply an answer.

See for example: https://gizmodo.com/googles-algorithm-is-lying-to-you-about-...

Re:Three areas where Google Search lags behind competitors: ... by tremon on Wednesday April 13, 2022 @05:49PM (#31020723)
But "Mars" is not simply a misspelling of the correct object, it confuses two distinct objects. A better comparison would be "When did Louis Armstrong set foot on the Moon?"
Re:Three areas where Google Search lags behind competitors: ... by JoshuaDavid on Wednesday April 13, 2022 @06:22PM (#31021023)
Amusingly, "what date did Louis Armstrong land on the Moon" displays the same behavior of showing the date "July 20, 1969" and no indication that it answered a different question than the one you asked.
Re:Three areas where Google Search lags behind competitors: ... by tshaddox on Wednesday April 13, 2022 @05:54PM (#31020774)
Indeed, it's a confusion between two distinct objects, and the intended object is overwhelmingly clear, which is why it's appropriate to correct the confusion and give the intended response.
Re:Three areas where Google Search lags behind competitors: ... by WestCoastJustin on Wednesday April 13, 2022 @05:20PM (#31020428)
It's correct for me. What are you getting?

Edit, yes I'm wrong, did a mental s/mars/moon/ without noticing.

Re:Three areas where Google Search lags behind competitors: ... by lhorie on Wednesday April 13, 2022 @05:22PM (#31020465)
The date when he landed on the moon. For mars, the correct answer is "never".
Re:Three areas where Google Search lags behind competitors: ... by WestCoastJustin on Wednesday April 13, 2022 @05:25PM (#31020494)
Ah, I totally misread that. Hilarious. Brain auto-correct strikes again.
Re:Three areas where Google Search lags behind competitors: ... by spookthesunset on Wednesday April 13, 2022 @07:31PM (#31021605)
I suspect this happened to a lot of people. I read it like “is the name misspelled?”
Re:Three areas where Google Search lags behind competitors: ... by quickthrowman on Wednesday April 13, 2022 @05:56PM (#31020793)
Google search doesn’t know the correct answer, but it does know what people usually search for and returns an answer to that. It’s silently adjusting Mars to Moon before returning an answer.
Re:Three areas where Google Search lags behind competitors: ... by drusepth on Wednesday April 13, 2022 @05:22PM (#31020455)
I get July 20, 1969. Based on the results that follow, I assume that is correct.
Re:Three areas where Google Search lags behind competitors: ... by cookiengineer on Wednesday April 13, 2022 @05:25PM (#31020498)
It is not. There was no human on Mars (yet) and that's the point of the question, proving that google's ML approach will recorrect and reinterpret the question and give a wrong answer.
Re:Three areas where Google Search lags behind competitors: ... by asdfasgasdgasdg on Wednesday April 13, 2022 @05:38PM (#31020622)
Although it's interesting to note that two presumably human commenters in this thread got that wrong. Maybe the Google results are more human than we would have guessed.
Re:Three areas where Google Search lags behind competitors: ... by dekhn on Wednesday April 13, 2022 @05:49PM (#31020724)
it's more showing that the search engine isn't much better than bag-of-words model, and doesn't seem to "know" enough logic/reasoning to parse a sentence and determine that it's false because it says mars instead of the moon.

BTW, a big reason for this is the search quality folks at Google left the building and got replaced with growth marketers.

Re:Three areas where Google Search lags behind competitors: ... by asdfasgasdgasdg on Wednesday April 13, 2022 @06:50PM (#31021282)
This narrative doesn't really ring true to me. In any case, if you're saying that this result indicates that the system is no better than a bag of words model, doesn't that indicate that humans are equally no better, considering the errors made by the two commenters in this thread?

To me it just seems like a query that is likely to trip any imperfect entity up, since a "when" question usually implies that the event in question is known to have happened.

Re:Three areas where Google Search lags behind competitors: ... by htrp on Wednesday April 13, 2022 @06:14PM (#31020967)
>BTW, a big reason for this is the search quality folks at Google left the building and got replaced with growth marketers.

Is this the transition they made to AI powered rankings a couple of years ago?

Re:Three areas where Google Search lags behind competitors: ... by jeffybefffy519 on Wednesday April 13, 2022 @05:25PM (#31020495)
I love that Google shills will defend that Instant Answers is a good idea. Its just a disinformation weapon to bad guys.