News for hackers. Stuff that matters.
Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @01:23PM (#34712370)
You know, whatever the merits (and it sounds as if there were very few) of Google's presentation (or Pinchai's recent non-statement) might be, I would have expected the commentariat here at HN to display way more skepticism about language models as a replacement for search. If there's a crowd anywhere in the world large enough to leave 200+ comments on a post like this, you'd think thatg the one at HN would understand these models' currently unsolved problem with lying and fabulation.

And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident? The performance of these models is simultaneously jaw-dropping and absurd. With no proposed solutions to the problems that they fundamentally face, why this level of belief that their ascendance is inevitable?

Re:Google's Live from Paris Event Private/Deleted Immediately by rchaud on Wednesday February 08, 2023 @01:40PM (#34712651)
> And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident?

It is difficult to get a man to critically evaluate something if his future salary depends upon his unquestioning acceptance of it.

Re:Google's Live from Paris Event Private/Deleted Immediately by red-iron-pine on Wednesday February 08, 2023 @02:27PM (#34713470)
SV and Wall Street loves a smart guy who bucks the trend, big risk taker.

Mostly a question of pucker factor -- how big of a bet in favor or against, and how big of an impact on the job will you risk?

Re:Google's Live from Paris Event Private/Deleted Immediately by rchaud on Wednesday February 08, 2023 @02:38PM (#34713673)
> SV and Wall Street loves a smart guy who bucks the trend, big risk taker.

Only if they hit out of the park. You don't see a lot of founders getting a second bite of the Sand Hill Road apple if they failed the first time.

Re:Google's Live from Paris Event Private/Deleted Immediately by sho_hn on Wednesday February 08, 2023 @01:26PM (#34712419)
I'd like to have more discussion about the attribution problem, specifically.

We now have a couple of the players saying they're working on, or having demos. But from what I can see, all of these don't _actually_ attribute the source. They're just able to find _a_ source that fits to the output, working backward from it. Often the attributed source fits, but is actually disagreeing with the original output in specific details. In other words, it's not backing the output at all. At most it's a "you could check out these links and compare yourself and it might help you judge our accuracy".

Is there any indication real attribution is coming?

Re:Google's Live from Paris Event Private/Deleted Immediately by rchaud on Wednesday February 08, 2023 @01:42PM (#34712691)
> Is there any indication real attribution is coming?

I wouldn't hold my breath. Showing attributions allows people to reverse engineer how the prompt was made, and identifies weaknesses in the model. Bad for business!

Re:Google's Live from Paris Event Private/Deleted Immediately by sho_hn on Wednesday February 08, 2023 @01:47PM (#34712773)
I feel like all of the interesting stuff when it comes to search+LLM is in real hybridization that's still a few steps out from what's happening right now in terms of productization.

For example:

(a) Addressing the recency problem by making a model generate instructions to a browser as output token stream, so it can, well, Google for you and incorporate latest info into output by adding it to the "internal prompt".

(b) Doing something similar where you teach models how to drive computational engines to make actual, computed facts and figures, not guess them from statistics over training data.

(c) Maybe running a traditional search and then using LLMs to summarize over them.

etc. etc.

It feels increasingly clear that "complete prompt from a stale training data set" is a neat trick but doesn't solve any particular problem well enough. This also seems intuitively true: If you ask your friend a difficult question, you often don't expect them to be able to answer it on the spot. You probably expect them to have relevant experience and a rough understanding of the topics and the tools to look up or figure out the rest. Similarly, it feels intuitively wrong to expect a LLM to come ready-made with all knowledge baked in. It feels more like we'd want something with the working knowledge/method to plan and execute fact discovery and computation.

It's fun to play around a bit with ChatGPT yourself and try to approximate what a more complex pipeline might be. For example, instead of asking it to answer a question, ask it to re-state the question or make a list of questions prompted by the original question, then let it "contemplate" those or propose a method for how to go after answering them, etc. It runs aground pretty often, but sometimes the results take you a lot further than the direct approach.

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:00PM (#34712973)
> This also seems intuitively true: If you ask your friend a difficult question, you often don't expect them to be able to answer it on the spot. You probably expect them to have relevant experience and a rough understanding of the topics and the tools to look up or figure out the rest. Similarly, it feels intuitively wrong to expect a LLM to come ready-made with all knowledge baked in.

Not sure. In theory, this being computers and all, I sort of do expect my search engine to effectively come "ready-made with all knowledge baked in". I do not expect Google to start a new web crawl because I gave it a novel query, and I don't expect a language model (certainly not one being used for search purposes) to say "hang on, I need to go read some more before I answer that".

Re:Google's Live from Paris Event Private/Deleted Immediately by sho_hn on Wednesday February 08, 2023 @02:05PM (#34713041)
I've seen some studies that call what I mean a "chain of thought" approach (also reasonable is just "higher-level reasoning", but maybe we need a term for the engineered implementation), and I like that terminology. I don't know, it's a gut feeling, but a one-shot prompt completion feels intuitively too simple for me - it seems like we need to build up more elaborate "thought processes" next where we use these methods for individual steps in those thought processes, but also have steps that can carry out a computation or go on a fact search.

A bit similar, again, to how humans will answer plenty of questions on the spot from working knowledge, but are also able to classify when they need to drop into a slower "let's do the math" mode and then carry that out.

We may be able to from these "thought processes" then generate training data sets that again reasonably cache "all possible accurate answers" as you demand, but it's not clear to me if that caching will really save computational cost or just redistribute it to different times. At that point it's probably a latency/throughput question of when you do vs. defer the "thinking".

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:20PM (#34713343)
That's undoubtedly true for "reasoning". I don't think it is true for "search".

When I ask <search engine> a question that is "demonstrably" answerable based on "reliable" sources (e.g. what is the capital of Mexico), I don't need or expect it to reason. I want it to (a) tell me the most likely answer, probably (b) provide a list a of URLs that also provide an answer, probably the same one.

And that's what Google and others do right now; ignoring the results presentations being skewed by commercial interests, they do it rather well.

Figuring out when a question is not answerable in this way, but requires reasoning, is certainly a part of the challenge.

Re:Google's Live from Paris Event Private/Deleted Immediately by t-writescode on Wednesday February 08, 2023 @02:13PM (#34713214)
Look at the images in the announcement from yesterday: https://www.theverge.com/2023/2/7/23587454/microsoft-bing-ed...

Those look like citations, to me.

Re:Google's Live from Paris Event Private/Deleted Immediately by uoaei on Wednesday February 08, 2023 @02:04PM (#34713029)
What viability are top execs seeing in a business model that is so fragile as to be defeated by reverse-engineering these prompts...? There is obviously no competitive advantage here for very long. The systems must provide an additional service on top of these models, something beyond "attaching it to a publicly-available input form". It needs to aid in reasoning, not act as a god.
Re:Google's Live from Paris Event Private/Deleted Immediately by jerf on Wednesday February 08, 2023 @02:02PM (#34713003)
I was musing on my website about ChatGPT [1] and one of the things I noticed about it is that if you analogize it to your own brain, which has a language model in it too, we are collectively asking ChatGPT to do something we would never ask of our own language model. ChatGPT does not have the "higher level" reasoning that we're asking for, to do logic, attribution, and the other things we want.

This is not necessarily a "criticism". It is obvious that ChatGPT produces an incredibly rich and meaning-filled representation of the text, as the extension of the text it can be used for demonstrates. But we should not be trying to "tweak" the language model here and there, trying to make a language model do something it isn't really suited for. We should be working on how to hook up that higher-level functionality. It is not obvious how to do that, true. But it at least stands a chance of producing the AI in reality that people think we have now, but don't.

For that reason, I don't think attribution is coming without a significant addition to the architecture. Language models shouldn't be doing attribution; the mess you'd create of them to accomplish that would ruin them as language models. Can you imagine how insane you'd go if your internal language model tried to stuff down every instance of when you have seen and/or used the word "brain" every time you tried to use the word "brain"? That's not what language models are for. That's what the thing using the language model does. I expect AIs to parallel this. I see no reason at all, and abundant reasons to the contrary, to expect AIs to just be undifferentiated blobs of numbers. I expect them to have structures just as our brains do, and for the exact same reasons. If undifferentiated blobs of identical neurons was the way to go, our brains would work that way too; it's far simpler than what we actually have.

[1]: https://www.jerf.org/iri/post/2023/understanding_gpt_better/

Re:Google's Live from Paris Event Private/Deleted Immediately by sho_hn on Wednesday February 08, 2023 @02:17PM (#34713300)
Yep, this resonates heavily with my own thinking. Thanks!
Re:Google's Live from Paris Event Private/Deleted Immediately by Jensson on Friday February 10, 2023 @05:01AM (#34737706)
You don't remember where you learned the things you know from? How can you trust anything you think then? Can such people get anything done at all, seems basically impossible to learn new things then since you can't know if it is better than what you thought before?

So I don't think that remembering sources is a hard problem to solve, likely making the AI work well is probably easier if it remembers sources.

Re:Google's Live from Paris Event Private/Deleted Immediately by jerf on Friday February 10, 2023 @02:46PM (#34744751)
I don't think remembering sources is an intrinsically difficult problem.

I think it's hard for language models, and indeed, after further thought, I'm pretty sure they shouldn't remember sources: https://www.jerf.org/iri/post/2023/understanding_gpt_better/

But the problem in general is probably not a particular stopper.

A bigger problem is people thinking that we should be taking language models and using them as AIs themselves, rather than seeing them as a component of an AI, which the more I think about, the more obvious it is that this is the case. Language models aren't what we want. They're a necessary step to what we do want and I don't expect them to go anywhere. But history will look back and laugh at this misguided attempt to make them be the entire AI.

Also, if you are implying you can just spew out the exact sources you learned some fact from... no you can't. You think so only because you haven't tried. Ask someone around you for five random "facts" for you to cite your source from, then try to write on a sheet of paper where you heard this fact from for the first time. Then hand the sheet of paper back to your friend and have them verify your claims... so, note, immediately you have the problem that an unverifiable claim is a fail, so "in elementary school" doesn't cut it any more than you can cite "at the library" as a source on your high school essay. I'm not looking for words that "identify" a several year span of time. You need to give me book, page, and line, or timestamps on a video, or something like that. No, you can't. Nobody can just spit out a bibliographic citation from memory for when they learned that Pluto isn't a planet. I can give you "in the news" for that, sure, but what news source was first? What is the title of that article? Who wrote it? What was the exact date? Of course not.

Re:Google's Live from Paris Event Private/Deleted Immediately by telotortium on Friday February 10, 2023 @09:10PM (#34748864)
Steve Hsu has recently claimed[1] to have started a startup in stealth to solve the hallucination problem over a corpus as large as 10,000 pages of dense, college-level textbooks, to the point where it can answer the end-of-chapter questions in the textbook with almost 100% accuracy (albeit not math questions) - I hope without using the answer key. Not sure if their approach is more robust than Supabase[2] or similar approaches, and no indication of whether it might scale up to something on the order of a search engine corpus, but it's something.

[1] https://www.youtube.com/watch?v=peHkL_MaxTU&t=1558s [2] https://news.ycombinator.com/item?id=34695306

Re:Google's Live from Paris Event Private/Deleted Immediately by t-writescode on Wednesday February 08, 2023 @02:13PM (#34713205)
> Is there any indication real attribution is coming?

Looking at the main article on HN yesterday around it: https://www.theverge.com/2023/2/7/23587454/microsoft-bing-ed... you can see citations in the examples provided.

Re:Google's Live from Paris Event Private/Deleted Immediately by sho_hn on Wednesday February 08, 2023 @02:15PM (#34713262)
I've had those Bing images in mind specifically when writing the above, because I seem to recall them phrasing it very carefully as "you can look at these links for more information", not "we retrieved this info here".
Re:Google's Live from Paris Event Private/Deleted Immediately by vineyardmike on Wednesday February 08, 2023 @02:37PM (#34713651)
> But from what I can see, all of these don't _actually_ attribute the source. They're just able to find _a_ source that fits to the output, working backward from it

I feel like it should be possible to build attribution "simply" by finding good results via traditional search (or even using LLM embeddings), then asking a LLM to summarize the sources. Then you can show the attribution. This would (to me) be much more meaningful. Especially for specific searches ("I want a red oversized couch thats got good back support"). Don't tell me about couches then find a random link, find a couch then tell me about that couch.

Is anyone doing this? IDK, maybe there is a reason that it's not happening, but it seems at least as easy as all the other crazy (and cool) things happening in this space.

Re:Google's Live from Paris Event Private/Deleted Immediately by dhdaadhd on Wednesday February 08, 2023 @04:30PM (#34715488)
I think Neeva is doing this!
Re:Google's Live from Paris Event Private/Deleted Immediately by DOsinga on Wednesday February 08, 2023 @04:52PM (#34715853)
I think this what the demo's with sources do. But they might still run into the same problem - the LLM summarization might be a bit wrong and so now it is saying one thing and the source has a slightly different opinion.
Re:Google's Live from Paris Event Private/Deleted Immediately by nmca on Wednesday February 08, 2023 @06:04PM (#34716900)
GopherCite, Sparrow, WebGPT and most likely the new bing (WebGPT2) all first find sources then generate output, so you're just mistaken. The first two specifically get folks check whether the selected source snippet supports the answer.

Source: worked on GopherCite.

Re:Google's Live from Paris Event Private/Deleted Immediately by pasquinelli on Wednesday February 08, 2023 @01:54PM (#34712886)
> If there's a crowd anywhere in the world large enough to leave 200+ comments on a post like this, you'd think thatg the one at HN would understand these models' currently unsolved problem with lying and fabulation.

if i'm being frank, that doesn't match my experience with HN at all. chatgpt is the hot new thing, of course people here are preoccupied imagining a future powered with it.

there are a bunch of different types of people here, with different perspectives that might be at odds with one another, but they don't necessarily comment on the same articles.

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:24PM (#34713405)
in general, i often post similar comments on HN, disputing the idea that there is such a thing as "HN groupthink".

what puzzles me here is the the technical limitations of language models (i refuse to call them "large" because the sizes are going to change dramatically) seem so obvious and so well known, that i just find it hard to believe that there are more than a handful of people on HN who uncritically accept the idea that Bing is trying to sell.

Re:Google's Live from Paris Event Private/Deleted Immediately by pixl97 on Wednesday February 08, 2023 @02:55PM (#34713932)
With that said I'm not sure the cynics on HN are scoring much better (we've predicted the last 5000 of 10 major failures meme). The cynic in me said crypto was overhyped by con men and mostly useless, and I was right. At the same time I said the same about most social media and yet the masses flocked to it to give their personal information away.
Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @03:34PM (#34714594)
No worries, I was #2 at amzn and said it would never work :)
Re:Google's Live from Paris Event Private/Deleted Immediately by eric_cc on Wednesday February 08, 2023 @05:34PM (#34716505)
> disputing the idea that there is such a thing as "HN groupthink".

There 100% is HN groupthink. Any doubts about it should have been erased by observing HN’s position on bitcoin and cryptocurrencies.

Re:Google's Live from Paris Event Private/Deleted Immediately by uoaei on Wednesday February 08, 2023 @02:00PM (#34712965)
Any numerical system operates only on numerical, not logical/grammatical/whatever constraints. Wrongness is built into any system based solely on neural components by the simple fact that rightness is unverifiable from within these systems' own "logical" (read: associational) apparati.

I share your confusion regarding the grave dearth of incredulity in the reception of these technologies.

Re:Google's Live from Paris Event Private/Deleted Immediately by jakeinspace on Wednesday February 08, 2023 @02:12PM (#34713184)
I guess you can say the human brain operates only on the data it gets from our ear drums, optic nerve, etc. Might as well call that numerical input, it’s certainly not grammatical or logical, all that is internal and arises after many layers of neurons.
Re:Google's Live from Paris Event Private/Deleted Immediately by uoaei on Wednesday February 08, 2023 @08:02PM (#34718260)
I agree: the human (or any organic) brain is no more "logical" than any other numerical system. We obscure this fact (mostly) successfully by virtue of the statistical regularities with which we are imprinted as a result of our experiences. This gives the appearance of logical rigor, but this rigor is only illusory.
Re:Google's Live from Paris Event Private/Deleted Immediately by zzzeek on Wednesday February 08, 2023 @02:12PM (#34713186)
simple answer is the consensus groupthink on Hacker News is deeply wishful

i made this same point at https://news.ycombinator.com/item?id=34699087 and got nothing but pushback how there is no real problem with ChatGPT and Bing just fixed all the problems (which didn't exist) anyway.

Re:Google's Live from Paris Event Private/Deleted Immediately by tarsinge on Wednesday February 08, 2023 @02:17PM (#34713296)
That thread is mostly about Google completely messing up their response to what Microsoft announced yesterday.

I don’t see that unanimous acceptance, there are a lot of interesting discussions about the shortcomings. There are many comparisons between Google results for examples, with also potential solutions to problems you cite like https://news.ycombinator.com/item?id=34709883

It’s not about replacing search overnight, but if it can take even 10% of search volume that would have serious consequences for Google.

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:28PM (#34713477)
If it takes even 10% of search volume (in its current technological form) then those of us working in this domain have failed, miserably.

I have no idea what Microsoft is thinking. It's like a state hiring a known liar to be its star witness in some notable number of court cases.

[ EDIT: OK, so one thing they might be thinking is to just this sort of technology to generate readable/listenable summaries of search results obtained by other means. I can see much less wrong with that, but it's also not really the same technology at all as ChatGPT et al. ]

Re:Google's Live from Paris Event Private/Deleted Immediately by pixl97 on Wednesday February 08, 2023 @02:59PM (#34713984)
Heh, this is where I think you may be wrong about the future....

We vote in known liars all the time. People tend to like a good story better than the truth. If this thing is good at storytelling we could be in trouble.

Re:Google's Live from Paris Event Private/Deleted Immediately by jjkaczor on Wednesday February 08, 2023 @05:24PM (#34716347)
...It's like a state hiring a known liar...

Even worse than a star witness, the US let a known liar run their entire country for 4 years... So, frankly... people will love ChatGPT and other similar clones - people love bullshit.

Re:Google's Live from Paris Event Private/Deleted Immediately by 1vuio0pswjnm7 on Wednesday February 08, 2023 @02:19PM (#34713336)
Assuming that Google is in a panic over ChatGPT, as suggested the HN commentariat and the "tech" news media, then arguably this only highlights how little confidence Google has in the quality of its search engine and the trustworthiness of the web as a source for information. For example, instead of trying to match or beat ChatGPT, Google could focus on illustrating the shortcomings of conversational AI versus a search engine as a reliable and trustworthy means of locating information on the web.

As usual, it appears "tech" companies have an extremely warped view of web users as being non-discerning consumers of whatever garbage "tech" companies feed to them.

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:34PM (#34713596)
Absolutely. In part, my comment targets Google itself as much as the HN commentariat, since the corporate communications, as you note, suggest a distinct lack of confidence in what they already do, and a great deal of over-confidence in what LM's might bring.
Re:Google's Live from Paris Event Private/Deleted Immediately by voisin on Wednesday February 08, 2023 @04:38PM (#34715616)
> arguably this only highlights how little confidence Google has in the quality of its search engine and the trustworthiness of the web as a source for information.

I think they are realizing they killed their golden goose by filling it with trash ads and it only worked in the absence of something more accessible to the public. They were drunk at the wheel and crashed into (for the first time ever) a competent competitor. Now they realize they need to sober up quickly but their organization is lazy and bloated, they’ve taken their users for granted and abused them at the altar of advertisers, and generally done nothing for two entire decades to garner any form of loyalty.

They’ve got some work cut out for them. From my perspective, Google could come out with a ChatGPT killer and I still wouldn’t use it, because the company has a track record of arbitrarily killing off any product I’ve actually liked.

Re:Google's Live from Paris Event Private/Deleted Immediately by jozvolskyef on Wednesday February 08, 2023 @02:25PM (#34713434)
The disruption isn't hypothetical. People like me are excited about the technology because it already has disrupted search for us. My brother has been using it for the majority of his queries because he doesn't speak English and ChatGPT is a gateway to the knowledge that only exists on the English-language internet.
Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:33PM (#34713581)
I wasn't disputing that the disruption exists. I was questioning whether it should exist, given the technological limitations of LMs at this time.
Re:Google's Live from Paris Event Private/Deleted Immediately by potatolicious on Wednesday February 08, 2023 @02:27PM (#34713465)
Agreed heavily - and I work with these technologies regularly. There is a mania around it much like there was for crypto (though I would maintain there's more merit here than there ever was with crypto - orders of magnitude more).

There are a litany of problems that surround LLMs that are solvable, but are highly non-trivial. The present hyped demos all seem to involve mainlining the unfiltered output of LLMs directly at the user, and this approach strikes me as largely impractical, with all of the issues you've brought up and more.

Ultimately I think we will land on a few learnings, some of which are IMO pretty obvious already with existing ML:

- There are going to be lots and lots of use cases for LLMs where despite its additional complexity it won't out-perform simpler ML models, or even heuristic-based solutions. This has been a plague for AI startups forever - where the shameful truth is that the ML bits do not significantly outperform heuristic-based approaches, but the ML bits sure as useful for the hype machine. There will be more of these use cases than use cases where LLMs actually do significantly move the state of the art forward.

- In most broadly mainstream-palatable applications the LLM will need to be intermediated by many other systems - some using other ML models, and some using heuristic-based solutions. For example for search, mediating the output through a knowledge graph with reliable provenance and accuracy. Realistically there will be vanishingly few use cases that are thin-wrappers around LLMs. Real usefulness of this tech will require very heavy lifting around the core model to make it actually accurate/reliable enough to be useful on a mass scale. The many products spinning up that seem to be premised on very thin wrappers around OpenAI's API will IMO mostly not survive, both because there is no moat and because the results won't be sufficiently useful.

- Accuracy matters and will be a thorn in the side of these products for quite some time, even though it is solvable. For some use cases (mostly creative ones with a human in the loop - like creative writing or image synthesis) the lack of accuracy is livable, but for most others the propensity to hallucinate is going to be a major barrier to adoption. Right now you're dealing with an enthusiast audience that's willing to look past glaring and obvious errors in output in favor of what it could be and how far we've come, but a mainstream audience I suspect will be less forgiving. Social media virality around product failure cases will also further punish products.

Overall I think what we're looking at here is legitimately a huge leap forward, but this legitimate and well-found excitement needs to be tempered by the fact that some (most?) players in the space seem to be getting in way over their skis, and that truly productionizing this technology is going to be intensely difficult.

[edit] To be a bit less doom and gloom about this - I think the real winner here are natural language interfaces, and stepping ever closer towards computer systems that do not need to be actively learned before being used. There is IMO an over-focus on the knowledge-encoding part of LLMs (which is unreliable and prone to hallucination) and not enough focus on the language-encoding part of LLMs (which is what they actually do). For the most part voice/language-based interfaces have had limited traction because they honestly haven't been smart enough to understand user requests with a sufficient level of expressivity and fidelity. I suspect these products (and new ones) will have another crack at this problem.

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @02:36PM (#34713631)
Great comment, thanks. However:

> but a mainstream audience I suspect will be less forgiving.

Less forgiving, perhaps, but also, I suspect, less discerning.

Re:Google's Live from Paris Event Private/Deleted Immediately by potatolicious on Wednesday February 08, 2023 @03:41PM (#34714725)
Initially sure, but that's what media (formal, social, and otherwise) is for. All you need is for the thing to blow up spectacularly in a way that goes viral and your product is now tainted with "unreliable", "wrong", "makes shit up", etc etc.

Hell, it literally just happened today. Google's first public demo of Bard failed in a highly visible way in a highly public setting and that's a whole media cycle now and has sent the stock flying downwards.

Is this lethal for the product? I suspect not at this point - but if they can't wrestle this stuff under control it very well be.

Re:Google's Live from Paris Event Private/Deleted Immediately by labrador on Wednesday February 08, 2023 @02:31PM (#34713549)
Microsoft is doing it right. They display search results on the left and ChatGPT results on the right. It's up to you to decide what is accurate knowing the results on the right are supposed to be a summary of what is on the left.
Re:Google's Live from Paris Event Private/Deleted Immediately by ekanes on Wednesday February 08, 2023 @03:09PM (#34714154)
For me, it's that this recently-released product is already competing (in the minds of users, which is what counts) with a decades-old mature product. In terms of accuracy, the "but sometimes it's wrong" perspective reminds me of the initial reaction among some to wikipedia. Yep, ANYONE can edit it. Yep, they could lie... ¯\_(ツ)_/¯
Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Wednesday February 08, 2023 @04:41PM (#34715672)
There are and always have been mechanisms in place to address lying on Wikipedia, which work with varying degrees of effectiveness depending on the topic.

What is the mechanism to address lying by a language model?

Re:Google's Live from Paris Event Private/Deleted Immediately by rendall on Wednesday February 08, 2023 @04:51PM (#34715818)
> There are and always have been mechanisms in place to address lying on Wikipedia

Wikipedia's mechanisms are fundamentally unable to cope with motivated, coordinated gangs that conspire together to push a distorted perspective, particularly on niche articles. Given Wikipedia's prominence, this is a problem that will only get worse.

Re:Google's Live from Paris Event Private/Deleted Immediately by mistymountains on Wednesday February 08, 2023 @03:29PM (#34714507)
It’s difficult to disentangle people’s desire for Google to fail due to layoffs and corporate misbehavior from legitimate analysis about how LLMs will cause Google to fail.
Re:Google's Live from Paris Event Private/Deleted Immediately by mark_l_watson on Wednesday February 08, 2023 @05:52PM (#34716748)
I am not really disagreeing with you, but just wanted to add:

Probably the secret sauce to using LLMs for search and other forms of information retrieval is first taking a query and use traditional lookup to find likely information. Then aggregating this data, provide the aggregated data as context, append the query, and pass it to the LLM. Probably some post processing is a good idea also.

I used to do this with squad LLMs.

Re:Google's Live from Paris Event Private/Deleted Immediately by doomleika on Wednesday February 08, 2023 @08:09PM (#34718320)
I would argue the state of search engine(i.e. google) is worse than ChatGPT. Half of the time top of the search gives me non-answers like "it depends". I have to use `stackoverflow` `reddit` or incognito mode to force google at least give me something tangible, and I don't see google going to change in short term considering for them, me going through all 3 pages of search is engagement and ad revenue.

ChatGPT might not be ideal, but they are way better than current offering.

Re:Google's Live from Paris Event Private/Deleted Immediately by hahahawhat on Wednesday February 08, 2023 @10:42PM (#34719494)
"And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident? The performance of these models is simultaneously jaw-dropping and absurd. With no proposed solutions to the problems that they fundamentally face, why this level of belief that their ascendance is inevitable?"

Because you forgot the nr 1 rule of fight club. It doesn't matter if it has unsolvable problems. The only thing that matters is if it has the momentum and the support to create a bubble and drive economic growth.

You're welcome.

Re:Google's Live from Paris Event Private/Deleted Immediately by hn_throwaway_99 on Thursday February 09, 2023 @11:16AM (#34726259)
> I would have expected the commentariat here at HN to display way more skepticism about language models as a replacement for search.

Because plenty of people don't see it necessarily as a replacement for search, but as a useful adjunct.

More to the point though, why do people always want to define the "HN commentariat" as some monolithic bunch? I've seen tons of comments and articles on the front page pointing out many issues with LLMs, from attribution, to confident bullshitting, to bias issues. Heck, there was plenty of discussion about a big proposed reason for Google's stock crash yesterday was due to the ad Google used to highlight questions their LLM can answer gave a confidently incorrect response.

Whenever I see comments that treat the rest of the HN audience like some monolithic group, especially in the face of plenty of evidence to the contrary, I feel like, ironically, the poster is saying "Why don't you all agree with me?"

Re:Google's Live from Paris Event Private/Deleted Immediately by PaulDavisThe1st on Thursday February 09, 2023 @11:30AM (#34726533)
I work really hard not to do this, and have more than once posted a message like yours.

However, my point here was that at the time of posting, the comment threads for this HN post were largely devoid of any skepticism about LM+search, and were completely dominated by a calm acceptance that it was all inevitable. And I wasn't trying to say "all of HN believes <X>", but rather "I would have expected there to be more voices in the comment thread expressing skepticism, because I know there are some smart people here"

Google's Bard demo certainly tweaked the situation by the end of the day.