But at the very least Google always gives you the source, which allows you to read it for yourself and judge its trustworthiness. No such option for ChatGPT.
1) Not be the actual source of content 2) Not link to the original source
Are we using the same search engine? I use DuckDuckGo most of the time but Google is hardly terrible, even if it could be improved.
I find what I’m looking for in the top results 90-95% of the time. The other 5-10% also contains times where there simply isn’t a good link that solves what I’m looking for even after digging hard.
Then you are not using the same engine!
(I also use DDG and find its results superior to google).
It's like how Twitter's default feed changed to try to tell you what you wanted to see, while the old chronological feed was the content you originally asked to follow. The simpler feed was always better and less contentious for non-newbie/casual users.
Google is better at deciphering what you want, but DDG is straight up: here's the simplest results based on your query.
Maybe the solution is both? An evolution of the ML approach that Google uses (ala ChatGPT style mimicking a human) AND offering a tab, or box, or whatever interface, with 'simple' straightforward old-school search results. Until the former is better 99% of the time, they shouldn't abandon the latter.
If Microsoft somehow is able to implement the information synthesis (e.g. "give me an example of C++ metaprogramming, using Boost Hana") with actual indexed links from sources, that would probably be the one thing I'd see in the near-ish term that could upset the search engine balance of power.
I would be really curious to see how trying to learn a new subject via a chatbot that is able to also provide source material links would go.
But an LLM doesn’t look at particular sources, find relevant ones, and write a response based on them, it is a mathematical summation of the way all of its sources look as sequences of text, and produces a response that based on that fits with its prompt. Identifying which sources most influence a response isn’t merely something that they don’t currently do, its information that, AFAIK, there is no plausible way to recover from the model.
A good chunk of the remainder of the times I click through - especially on work-related stuff - it turns out the page is some kind of cribbed content from GitHub, StackOverflow and so forth, but mangled together and reskinned with different randomly-generated usernames making the "comments", so it seems like it's a different site, and you can only really tell by picking a specific sentence that appears unique and then manually searching for that same sentence on several different sites. No clear source there either.
Web search is broken, and it has been for several years now.
Excellent: this quite quickly tells you that the search result was wrong. So you quickly got very relevant context and can tell at a glance that there is no source. You don't get any such context when GPT provides an invalid answer.
Undoubtedly he came off as clever, but being a harasser means that his achievements were outweighed by the benefits.
"he's a harasser => he must be a parasite on the organization"
because then you end up with
"he's provably an excellent worker and hugely productive => he can't be a harasser, you're mistaken/lying"
Some people are really smart and productive and a credit to their profession and also awful to women and/or minorities, and you'd never know if you didn't witness/experience it.
But if they are awful only to white straight males, they are ok?
Harassment is harassment, stealing someone's work is stealing someone's work, let's keep changing the meaning of words to politicians.
Next year will see AI begin to disrupt so many businesses you'd thought were absolutely impenetrable. I don't think anything is safe anymore.
By the end of the decade, I think even manufacturing and robotics will be automated. Everyone from McDonalds to Apple and Ford will feel the pressure.
I've got a mind-blowing demo coming out soon that will put even more fields on notice. And that's just my team. There are so many people working on AI-first products now. Incumbents will be overwhelmed and washed away.
(If ever there was a cue for a career change or opportunity to make a whole lot of money, it's this. Don't ignore this one! You're still early enough to take advantage.)
I can think up architectures I could draw on a napkin that could do this. Focused research teams will nail this.
It's coming.
Any suggestions on how to take advantage?
Dive into ML development?
Learn how to write prompts for AI engines?
Invest in AI companies? Sadly OpenAI is private, but Microsoft is a monetizing partner I guess (Copilot)
Details?
Now of course humans want and need more than just 3 meals but still pretty good.
But if most of your work consist of interfacing with computers, you might be in trouble.
It’s very rare for automation to cause unemployment to the point it possibly never happened.
Why, because of LLMs in particular? What is the ground-breaking research that will be fueling an AI revolution?
> There are so many people working on AI-first products now. Incumbents will be overwhelmed and washed away.
Incumbents like Google have been doing AI for ages. Microsoft has a very important investment in OpenAI
> By the end of the decade, I think even manufacturing and robotics will be automated. Everyone from McDonalds to Apple and Ford will feel the pressure.
McDonald's, Apple or Ford sell products, not manufacturing services. They will be delighted if manufacturing is automated.
Please notice that in current setup of 'Internet capitalism' people that produce value and content earn nothing, and people that harvest that content for profit or engage in outright fraud make fortunes.
It's a lot like iRobot or other stories in which the robots will start to subtly misbehave, like a program you can't debug.
Dark ages where millions died in pointless wars over who's imaginary diety is the true one. A third of europe dying of preventable plague. It took a rigorous system of citation and a publically funded system of education to crawl out of the drakness.
Should we try AI powred dark ages but with resource shortage and nuclear weapons?
Maybe it'll learn to adopt a new view, but for now it won't be through actual reasoning.
(The most likely answer is the letter “e”.)
If I watch a youtube tutorial teaching me how to bake pizza and it comes out totally wrong and a watch another that produces exelent pizza, I disregard the first channel.
And there are many more ways where humans check the reputation of sources automatically.
GPT currently lacks this possibility.
What if we invented an algebraic notation for search indexing and allowed people to index and classify documents in their own way. Then a search engine could do something special with all that.
Human intelligence cannot progress without a notation to capture our thoughts in.
My favourite Google extracts:
Q: Who invented Running
A: [random guy] invented running in 1865 when he tried to walk twice at the same time. It was an overnight success
Q: How much does a [model of a bycicle] weight?
A: 140 KG (Maximum weight of the rider)
Q: Define an english Person
A: A Cunt
But chat bots could be designed to do that, no?
Are there any siginificant technical barrier to prevent so?
A revolution in this area would require some entirely new training algorithm that can reach GPT-levels of fidelity while using magnitudes less training material. An even bigger GPT model trained on even more text is not going to do much in approaching a solution to this problem.
I have read many similar comments to the grandparent, all saying the same thing, “I tried it, and it turned up wrong facts” all the while they forgot that they were deeply engaged with the content it was producing. Just think of how much our attention is monetized already, money will be poring into this space from here on. It’s only a matter of time.
Producing hallucinations or not isn't just a setting you can tweak in the model. The entire function of GPT-3 is to guess the next word from the words that came before. It's a hallucination machine that has been trained on so much real-world data and has such a large parameter set that for commonly known and discussed information it does a remarkably good job at creating factual sentences. But as soon as you get out into territory that it doesn't have memorized, it will do the next best thing and produce credible-sounding new material that may or may not be complete nonsense. Again, this isn't a setting you can tune, it's just what a transformer does. It models human language, not all of human cognition.
Also, since we can watch the model evaluation and see all its weights, there’s research that shows you can determine if any answer is a “retrieval” of existing knowledge or a hallucination.
For example, I asked who the first president of Argentina was. It gave the wrong name (Juan Manuel de Rosas) with a half-true bio. I then asked it for books where I can learn more about this person.
It produced five books. None of the book-author pairings were correct, though three of the books were real and four of the authors actually wrote books. Of the four authors that existed, three actually did write about Argentina's history, while one is a professor of Spanish and gender studies.
This is my point: ChatGPT was trained to produce text, not to produce facts. What I'm seeing here isn't just something that can be optimized away, it's fundamental to the model's design. It was basically told to produce text that looks like it could have been written on the internet. Its text is really convincing: when I saw the list of books I thought for a minute it had actually done it! But it's still just inventing text that reads well, because that's what its training was optimizing for.
It could be paired with a different system that searches for facts, but then it's just a friendly abstraction layer on top of Google or some other system, not a revolution in search.
Other than that it betrayed no understanding of the material. It rather resembled a high school kid who had read a few too many books and spouted platitudes about any and all subjects without being aware of what was really going on.
And in either case, you're not solving the problem that commenters here are trying to solve: Google's bad results will still affect your search, so ChatGPT will be fed largely by dubious blogspam.
No, there’s a fundamental issue in that statistical language models don’t start with facts to form sentences, they form sentences based on previous tokens and statistical weights. The weights are not auditable or easily updatable.
The lucky thing for Google if they got it working, is they are half there already. As much as I despise Google, I have to admit GPT won’t be their death.
[0] https://support.google.com/knowledgepanel/answer/9787176?hl=...
That's only because it was trained on non-facts also.
But that holds for Google Search just as well ...
Google Search gives you links to content that might have the answer (excluding their own box). You can see the differences between the authors of said content.
ChatGPT gives you content that looks like the answer. You can't see the authors of the generated content.
There's also the ethical question. For example, duckduckgo will show you stack overflow answers. Google could easily do this but they have already gotten flack for displaying answers and taking away clicks from sites, which means reduced ad revenue for the resource actually being used. We could make even stronger arguments about services like ChatGPT in their current form, let alone the one we're talking about.
If we think a bit further in the future there's another extension that might not be what we want. Suppose our model contains all current human knowledge and can accurately give you answers and even produce paragraphs at will. This kills off a lot of websites. Is this a good thing? Suppose we incorporate an image generator along similar lines and now we have text and image search. That's a huge centralization force. There are a lot of great benefits to this that I don't want to downplay, but we should also think about if this is actually what we want because there are downsides too.
I thought I should also add that I had an at length discussion of these LLMs being Stochastic Parrots (an how we know they are) in a thread from 2 days ago https://news.ycombinator.com/item?id=34113011
So I think if you want to see more people with my dream then push towards a post scarce world. We'll still have plenty of resource constraints, but it does change the game a lot. I think you'll see more artists as well. A new renaissance. But I fear the transitionary period might be quite difficult and disruptive. I do think such a future has a fairly good probability of happening within many of our lifetimes and almost guaranteed if we actively pushed for it (or solved adjacent problems like HLI).
So don't be so pessimistic. There's millions of people out there that are interested in pushing the bounds of human knowledge because that's what they want. I'd argue that physicists and mathematicians wouldn't exist if this weren't true. They get paid shit for extremely difficult work. We could say the same about many sciences. While not all of these people are in it for bettering humanity (just for pursuing knowledge) it often aligns and there is a decent portion who believe this is the point of science and our purpose, as humans, in life.
I don’t think it is.
> Why can’t it have a mode where it returns more rigorous results, and includes some system of links as citation?
Because it is a statistical text-generation model; connection of results to specific sources is fundamentally not part of the way it works, so it can’t just be asked to supply them. You’d need a fundamentally different approach to do that.
In any case it can be trained to assert confidence via a side channel
I just did a discussion with ChatGPT on intuitionistic logic and it gave answers that were subtly wrong in my understanding of the subject. I have an MA in math but I couldn't be sure whether my impression of its wrongness was correct until asked it for a reference and it gave a nonexistent book by Per Martin-Lof, "Intuitionistic Logic", along with a list of actual books by him. It's an impressive and scary bullshit generator at this point.
And if someone can find a reference to Intuitionistic Logic by Per Martin-Lof, Oxford University Press, 1982, well, I'm wrong. But I went through Wikipedia, Google, Worldcat and Google Books.
And it's definitely less prone to obvious bs than just a few weeks ago. For a lot of general things, it gives a summary paragraph that's seems a distillation of standard information - the main problems seem to appear when these "scripts" lack consistency.
Edit: Once it "learns" to get citations right, you'll have to dig into those to see if it's right. It's easy to see that arms not looking good.
But that's fundamentally not possible for the model as is, not in the sense you would normally think of citations at least. If it were to give an honest answer on "why did you say that Obama was born in Chicago", the model would generate training set example with propositions like Obama was born in ____, ___ was born in Chicago, ____ was ____ in ____ and so on - because that is how it learns.
Instead, to generate what you would actually want as citations, it would have to have a way to look up the statement it just produced in a corpus of "citeable materials" and find the closest match, based on some entirely different ML model most likely (presumably something similar to Google's search algos).
Which is the exact same issue that plagues it for the answer it’s generating a citation for.
I believe that it can be trained to improve this, and I believe it can be nearly solvable but I don’t believe it’s a solvable problem.
It can be trained to get facts mostly right using old-school academic tests and adversarial/supervised training, like an elementary school child. Today, it (supposedly?) punts math and some other logix to a separate function, and it just compiled the prompt. That can work for validating citations and “known facts” in many cases.
But again, it can’t “think” and understand when it returns something illogical (regardless of fact correctness).
Those people are wrong; statistics are a kind of reasoning (intuition) and if you try it, it obviously does the other kinds of reasoning too. I have all kinds of answers from it that are logically coherent and can’t possibly have appeared on the internet already.
(Try asking it to write new SCP wiki entries.)
You can see it “does” reasoning if you say “think step by step” after a question. It can attend over its steps then.
What are you basing this assumption on? GPT can always generate a bogus answer, because its only notion of whether an answer is good or not is whether your prompt + that answer are statistically similar to text in its training corpus. The more text from the training corpus it resembles the better GPT will think the answer is. For more obscure topics, this means that the chances of it hallucinating based on similar sounding much more popular topics increases greatly.
This way of working also means that facts that appear more in the training corpus have a higher chance of being part of the model's outputs. For example, if the training corpus was not carefully curated to exclude vaccine conspiracy theories, there would be a good chance that the answer GPT gives to "autism is caused by ___" would be "vaccines", as this would likely be the most common answer in its training corpus. In contrast, Google's algorithms have more sophisticated biasing especially for these kinds of sensitive topics to ensure that more high-confidence sources are likelier to be top results, even if they are a minority of all the pages on the internet on that topic.
No, that is false. “Statistically similar” is doing a ton of work even for the original GPT model - as I said somewhere else, just printing the letter “e” is “statistically similar” to English text, if it can even be grammatically correct you should be surprised.
And it’s certainly not true for the last two generations (InstructGPT and ChatGPT) which weren’t trained with that objective at all but used human raters.
This is a different model, called GPT-3, released in 2020. ChatGPT has the same number of neurons, but is trained with both human feedback and web data, rather than just web data like GPT-3.
What you're saying still holds up, though -- ChatGPT still hallucinates information.
Verification of facts requires direct experience through careful experimentation.
ChatGPT’s responses can either be verified or they are as unfalsifiable as any other confidently expounded conjecture found on your computer screen.
These models have no way to differentiate that elves are fictional while Indians actually exist.
This doesn't mean that it can't produce accurate facts, most of the time it does! But when it does produce nonsense, it does it in exactly the same tone of authority, so if you don't already know the answer you may well walk away believing an AI hallucination.
And the trouble is it doesn't really matter if everyone here thinks "well, I would follow up each request with research to verify the answer", because most people won't! This is like the Google answer extracts, which fairly frequently mislead by extracting out-of-context quotes, except that there's no way to get the original context and there may in fact be no original context! This makes follow-up research much more complicated than with Google and therefore unlikely to happen. If ChatGPT replaces Google, the amount of nonsense on the internet will get even worse, which is something that until 2022 I never thought was possible.
[0] https://github.com/minimaxir/gpt-3-experiments/blob/master/e...