As someone pointed out, it appears that Gemini was regurgitating fanfiction about Encanto 2: https://bsky.app/profile/russellsilva.bsky.social/post/3lef3...

I'm not sure I'd call this a hallucination. With the prompt being simply "Encanto 2", I'm tempted to say that this is working as intended, Gemini is giving the user what information is available on the topic. Of course, it could have shown better judgement that this was not correct information about the actual sequel.

It's much worse than a hallucination - it's referencing a link to a Fandom wiki dedicated to fan ideas.

What this means is Google AI search is failing to accurately vet context as well as relevance when selecting sources.

Hypothetically, you as an SEO could exploit this behavior to ensure biased info is represented by trying to present information as "authoritative" and "neutral"

But do we see the big tech companies doing anything to stop this kind of thing happening? Or is it the case of "she'll be right" attitude?

Are the big tech still running this as a big beta test, without telling people?

Personally I ignore anything until the first search result. And I disable/hide anything AI-related in results. It's bad enough with the load of AI-generated SEO spam going around.

In the meantime, do we tell people not to trust AI-generated content in search pages, opening actual search result links instead?

> But do we see the big tech companies doing anything to stop this kind of thing happening

Engineering's hard and SEO is a game of cat-and-mouse.

While Google Search has absolutely become worse due to their own choices, SEO Players have also become much more adept and sophisticated.

If OpenAI didn't rush ChatGPT to market, most players would have probably spent more time working on these kinks.

Has anyone been saying the AI results aren’t beta or experimental? It’s been all over the place that I’ve seen.

But people used Gmail for a decade or so while it was beta too.

Do you expect them to wait until it’s perfect? Because that’s definitely not going to happen.

> Hypothetically, you as an SEO could exploit this behavior to ensure biased info is represented by trying to present information as "authoritative" and "neutral"

Actually, this is what the press and TV does. It's called "news"

I get a real kick out of the NPR member drives where they read out notes of why people are becoming members. They love to read ones that talk about how unbiased they are.

This isn't hallucinated. It is sourced from the links in the search results, and those sources are clearly cited. It incorrectly stated that the movie is real instead of speculative and fan generated.

Well I flagged the overview as inaccurate and now it no longer appears under that search. It's possible other people were doing the same thing though. As a general rule fanon (fan canon) wikis should probably be delisted as sources.

Personally I think these features at their best can be interesting to look at, though you have to take them with a grain of salt, and they aren't the magic sauce that makes search 'better'.