It is difficult to get a man to critically evaluate something if his future salary depends upon his unquestioning acceptance of it.
Mostly a question of pucker factor -- how big of a bet in favor or against, and how big of an impact on the job will you risk?
Only if they hit out of the park. You don't see a lot of founders getting a second bite of the Sand Hill Road apple if they failed the first time.
We now have a couple of the players saying they're working on, or having demos. But from what I can see, all of these don't _actually_ attribute the source. They're just able to find _a_ source that fits to the output, working backward from it. Often the attributed source fits, but is actually disagreeing with the original output in specific details. In other words, it's not backing the output at all. At most it's a "you could check out these links and compare yourself and it might help you judge our accuracy".
Is there any indication real attribution is coming?
I wouldn't hold my breath. Showing attributions allows people to reverse engineer how the prompt was made, and identifies weaknesses in the model. Bad for business!
For example:
(a) Addressing the recency problem by making a model generate instructions to a browser as output token stream, so it can, well, Google for you and incorporate latest info into output by adding it to the "internal prompt".
(b) Doing something similar where you teach models how to drive computational engines to make actual, computed facts and figures, not guess them from statistics over training data.
(c) Maybe running a traditional search and then using LLMs to summarize over them.
etc. etc.
It feels increasingly clear that "complete prompt from a stale training data set" is a neat trick but doesn't solve any particular problem well enough. This also seems intuitively true: If you ask your friend a difficult question, you often don't expect them to be able to answer it on the spot. You probably expect them to have relevant experience and a rough understanding of the topics and the tools to look up or figure out the rest. Similarly, it feels intuitively wrong to expect a LLM to come ready-made with all knowledge baked in. It feels more like we'd want something with the working knowledge/method to plan and execute fact discovery and computation.
It's fun to play around a bit with ChatGPT yourself and try to approximate what a more complex pipeline might be. For example, instead of asking it to answer a question, ask it to re-state the question or make a list of questions prompted by the original question, then let it "contemplate" those or propose a method for how to go after answering them, etc. It runs aground pretty often, but sometimes the results take you a lot further than the direct approach.
Not sure. In theory, this being computers and all, I sort of do expect my search engine to effectively come "ready-made with all knowledge baked in". I do not expect Google to start a new web crawl because I gave it a novel query, and I don't expect a language model (certainly not one being used for search purposes) to say "hang on, I need to go read some more before I answer that".
A bit similar, again, to how humans will answer plenty of questions on the spot from working knowledge, but are also able to classify when they need to drop into a slower "let's do the math" mode and then carry that out.
We may be able to from these "thought processes" then generate training data sets that again reasonably cache "all possible accurate answers" as you demand, but it's not clear to me if that caching will really save computational cost or just redistribute it to different times. At that point it's probably a latency/throughput question of when you do vs. defer the "thinking".
When I ask <search engine> a question that is "demonstrably" answerable based on "reliable" sources (e.g. what is the capital of Mexico), I don't need or expect it to reason. I want it to (a) tell me the most likely answer, probably (b) provide a list a of URLs that also provide an answer, probably the same one.
And that's what Google and others do right now; ignoring the results presentations being skewed by commercial interests, they do it rather well.
Figuring out when a question is not answerable in this way, but requires reasoning, is certainly a part of the challenge.
Those look like citations, to me.
This is not necessarily a "criticism". It is obvious that ChatGPT produces an incredibly rich and meaning-filled representation of the text, as the extension of the text it can be used for demonstrates. But we should not be trying to "tweak" the language model here and there, trying to make a language model do something it isn't really suited for. We should be working on how to hook up that higher-level functionality. It is not obvious how to do that, true. But it at least stands a chance of producing the AI in reality that people think we have now, but don't.
For that reason, I don't think attribution is coming without a significant addition to the architecture. Language models shouldn't be doing attribution; the mess you'd create of them to accomplish that would ruin them as language models. Can you imagine how insane you'd go if your internal language model tried to stuff down every instance of when you have seen and/or used the word "brain" every time you tried to use the word "brain"? That's not what language models are for. That's what the thing using the language model does. I expect AIs to parallel this. I see no reason at all, and abundant reasons to the contrary, to expect AIs to just be undifferentiated blobs of numbers. I expect them to have structures just as our brains do, and for the exact same reasons. If undifferentiated blobs of identical neurons was the way to go, our brains would work that way too; it's far simpler than what we actually have.
[1]: https://www.jerf.org/iri/post/2023/understanding_gpt_better/
So I don't think that remembering sources is a hard problem to solve, likely making the AI work well is probably easier if it remembers sources.
I think it's hard for language models, and indeed, after further thought, I'm pretty sure they shouldn't remember sources: https://www.jerf.org/iri/post/2023/understanding_gpt_better/
But the problem in general is probably not a particular stopper.
A bigger problem is people thinking that we should be taking language models and using them as AIs themselves, rather than seeing them as a component of an AI, which the more I think about, the more obvious it is that this is the case. Language models aren't what we want. They're a necessary step to what we do want and I don't expect them to go anywhere. But history will look back and laugh at this misguided attempt to make them be the entire AI.
Also, if you are implying you can just spew out the exact sources you learned some fact from... no you can't. You think so only because you haven't tried. Ask someone around you for five random "facts" for you to cite your source from, then try to write on a sheet of paper where you heard this fact from for the first time. Then hand the sheet of paper back to your friend and have them verify your claims... so, note, immediately you have the problem that an unverifiable claim is a fail, so "in elementary school" doesn't cut it any more than you can cite "at the library" as a source on your high school essay. I'm not looking for words that "identify" a several year span of time. You need to give me book, page, and line, or timestamps on a video, or something like that. No, you can't. Nobody can just spit out a bibliographic citation from memory for when they learned that Pluto isn't a planet. I can give you "in the news" for that, sure, but what news source was first? What is the title of that article? Who wrote it? What was the exact date? Of course not.
[1] https://www.youtube.com/watch?v=peHkL_MaxTU&t=1558s [2] https://news.ycombinator.com/item?id=34695306
Looking at the main article on HN yesterday around it: https://www.theverge.com/2023/2/7/23587454/microsoft-bing-ed... you can see citations in the examples provided.
I feel like it should be possible to build attribution "simply" by finding good results via traditional search (or even using LLM embeddings), then asking a LLM to summarize the sources. Then you can show the attribution. This would (to me) be much more meaningful. Especially for specific searches ("I want a red oversized couch thats got good back support"). Don't tell me about couches then find a random link, find a couch then tell me about that couch.
Is anyone doing this? IDK, maybe there is a reason that it's not happening, but it seems at least as easy as all the other crazy (and cool) things happening in this space.
Source: worked on GopherCite.
if i'm being frank, that doesn't match my experience with HN at all. chatgpt is the hot new thing, of course people here are preoccupied imagining a future powered with it.
there are a bunch of different types of people here, with different perspectives that might be at odds with one another, but they don't necessarily comment on the same articles.
what puzzles me here is the the technical limitations of language models (i refuse to call them "large" because the sizes are going to change dramatically) seem so obvious and so well known, that i just find it hard to believe that there are more than a handful of people on HN who uncritically accept the idea that Bing is trying to sell.
There 100% is HN groupthink. Any doubts about it should have been erased by observing HN’s position on bitcoin and cryptocurrencies.
I share your confusion regarding the grave dearth of incredulity in the reception of these technologies.
i made this same point at https://news.ycombinator.com/item?id=34699087 and got nothing but pushback how there is no real problem with ChatGPT and Bing just fixed all the problems (which didn't exist) anyway.
I don’t see that unanimous acceptance, there are a lot of interesting discussions about the shortcomings. There are many comparisons between Google results for examples, with also potential solutions to problems you cite like https://news.ycombinator.com/item?id=34709883
It’s not about replacing search overnight, but if it can take even 10% of search volume that would have serious consequences for Google.
I have no idea what Microsoft is thinking. It's like a state hiring a known liar to be its star witness in some notable number of court cases.
[ EDIT: OK, so one thing they might be thinking is to just this sort of technology to generate readable/listenable summaries of search results obtained by other means. I can see much less wrong with that, but it's also not really the same technology at all as ChatGPT et al. ]
We vote in known liars all the time. People tend to like a good story better than the truth. If this thing is good at storytelling we could be in trouble.
Even worse than a star witness, the US let a known liar run their entire country for 4 years... So, frankly... people will love ChatGPT and other similar clones - people love bullshit.
As usual, it appears "tech" companies have an extremely warped view of web users as being non-discerning consumers of whatever garbage "tech" companies feed to them.
I think they are realizing they killed their golden goose by filling it with trash ads and it only worked in the absence of something more accessible to the public. They were drunk at the wheel and crashed into (for the first time ever) a competent competitor. Now they realize they need to sober up quickly but their organization is lazy and bloated, they’ve taken their users for granted and abused them at the altar of advertisers, and generally done nothing for two entire decades to garner any form of loyalty.
They’ve got some work cut out for them. From my perspective, Google could come out with a ChatGPT killer and I still wouldn’t use it, because the company has a track record of arbitrarily killing off any product I’ve actually liked.
There are a litany of problems that surround LLMs that are solvable, but are highly non-trivial. The present hyped demos all seem to involve mainlining the unfiltered output of LLMs directly at the user, and this approach strikes me as largely impractical, with all of the issues you've brought up and more.
Ultimately I think we will land on a few learnings, some of which are IMO pretty obvious already with existing ML:
- There are going to be lots and lots of use cases for LLMs where despite its additional complexity it won't out-perform simpler ML models, or even heuristic-based solutions. This has been a plague for AI startups forever - where the shameful truth is that the ML bits do not significantly outperform heuristic-based approaches, but the ML bits sure as useful for the hype machine. There will be more of these use cases than use cases where LLMs actually do significantly move the state of the art forward.
- In most broadly mainstream-palatable applications the LLM will need to be intermediated by many other systems - some using other ML models, and some using heuristic-based solutions. For example for search, mediating the output through a knowledge graph with reliable provenance and accuracy. Realistically there will be vanishingly few use cases that are thin-wrappers around LLMs. Real usefulness of this tech will require very heavy lifting around the core model to make it actually accurate/reliable enough to be useful on a mass scale. The many products spinning up that seem to be premised on very thin wrappers around OpenAI's API will IMO mostly not survive, both because there is no moat and because the results won't be sufficiently useful.
- Accuracy matters and will be a thorn in the side of these products for quite some time, even though it is solvable. For some use cases (mostly creative ones with a human in the loop - like creative writing or image synthesis) the lack of accuracy is livable, but for most others the propensity to hallucinate is going to be a major barrier to adoption. Right now you're dealing with an enthusiast audience that's willing to look past glaring and obvious errors in output in favor of what it could be and how far we've come, but a mainstream audience I suspect will be less forgiving. Social media virality around product failure cases will also further punish products.
Overall I think what we're looking at here is legitimately a huge leap forward, but this legitimate and well-found excitement needs to be tempered by the fact that some (most?) players in the space seem to be getting in way over their skis, and that truly productionizing this technology is going to be intensely difficult.
[edit] To be a bit less doom and gloom about this - I think the real winner here are natural language interfaces, and stepping ever closer towards computer systems that do not need to be actively learned before being used. There is IMO an over-focus on the knowledge-encoding part of LLMs (which is unreliable and prone to hallucination) and not enough focus on the language-encoding part of LLMs (which is what they actually do). For the most part voice/language-based interfaces have had limited traction because they honestly haven't been smart enough to understand user requests with a sufficient level of expressivity and fidelity. I suspect these products (and new ones) will have another crack at this problem.
> but a mainstream audience I suspect will be less forgiving.
Less forgiving, perhaps, but also, I suspect, less discerning.
Hell, it literally just happened today. Google's first public demo of Bard failed in a highly visible way in a highly public setting and that's a whole media cycle now and has sent the stock flying downwards.
Is this lethal for the product? I suspect not at this point - but if they can't wrestle this stuff under control it very well be.
What is the mechanism to address lying by a language model?
Wikipedia's mechanisms are fundamentally unable to cope with motivated, coordinated gangs that conspire together to push a distorted perspective, particularly on niche articles. Given Wikipedia's prominence, this is a problem that will only get worse.
Probably the secret sauce to using LLMs for search and other forms of information retrieval is first taking a query and use traditional lookup to find likely information. Then aggregating this data, provide the aggregated data as context, append the query, and pass it to the LLM. Probably some post processing is a good idea also.
I used to do this with squad LLMs.
ChatGPT might not be ideal, but they are way better than current offering.
Because you forgot the nr 1 rule of fight club. It doesn't matter if it has unsolvable problems. The only thing that matters is if it has the momentum and the support to create a bubble and drive economic growth.
You're welcome.
Because plenty of people don't see it necessarily as a replacement for search, but as a useful adjunct.
More to the point though, why do people always want to define the "HN commentariat" as some monolithic bunch? I've seen tons of comments and articles on the front page pointing out many issues with LLMs, from attribution, to confident bullshitting, to bias issues. Heck, there was plenty of discussion about a big proposed reason for Google's stock crash yesterday was due to the ad Google used to highlight questions their LLM can answer gave a confidently incorrect response.
Whenever I see comments that treat the rest of the HN audience like some monolithic group, especially in the face of plenty of evidence to the contrary, I feel like, ironically, the poster is saying "Why don't you all agree with me?"
However, my point here was that at the time of posting, the comment threads for this HN post were largely devoid of any skepticism about LM+search, and were completely dominated by a calm acceptance that it was all inevitable. And I wasn't trying to say "all of HN believes <X>", but rather "I would have expected there to be more voices in the comment thread expressing skepticism, because I know there are some smart people here"
Google's Bard demo certainly tweaked the situation by the end of the day.
And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident? The performance of these models is simultaneously jaw-dropping and absurd. With no proposed solutions to the problems that they fundamentally face, why this level of belief that their ascendance is inevitable?