My understanding is that ChatGPT was trained on text from the Internet and public domain texts. There is orders of magnitude more text available to humans behind paywalls and otherwise inaccessible (currently) to these models.
Am I missing something?
In fact it will be interesting how much more it would be at copywriting for specific feilds once it can train on that data. I imagine an LLM trained on all that dusty text in courthouse basements would become a much better paralegal (won't be a lawyer I'm afraid) than vanilla chatGPT
Makes sense to use Transformers' data to train autonomous vehicles.
At my alma mater I remember the large-scale Google book scanning devices and what a herculean effort that was to digitize the largest university library system's books - University of Michigan - although only 7M texts from the entire collection of ~16 million texts: https://en.wikipedia.org/wiki/University_of_Michigan_Library) were digitized.I too was curious about the state of the Google Books project: https://www.edsurge.com/news/2017-08-10-what-happened-to-goo...
This is an interesting piece of ephemera from 2005, when Google started digitizing books at UMich: https://apps.lib.umich.edu/files/services/mdp/faq.pdf
As far as I recall, the Books project allowed the early n-grams functionality to be built out: https://ai.googleblog.com/2006/08/all-our-n-gram-are-belong-...
The Google Books Ngram Viewer tool is actually still in existence; you can play around with it here: https://books.google.com/ngrams/graph?corpus=0&content=Vorsp...
On the first quick read though, I thought to myself, "Can he use that sort of language here?"
Then I pictured Dr. Evil and it made more sense...
I mean ChatGPT 3 went viral and Google managed to ship Bard in a few weeks. I think the consensus is that ChatGPT is better, but it was literally sitting on the shelf ready to go.
I agree with your other points, but why would you think ChatGPT was not given all the data on the internet?
If you aren't storing the text, the only thing that stops you retrieving all the pages that can possibly be found on the internet is a small amount of money.
I'm pretty certain that OpenAI has a lot more than a small amount of money.
Even the paywall issue alone, I can’t see how they could scale doing paywall signups automatically. Each paywall form is different, may require a local phone number in a different country to receive a text, etc.
[0]: https://github.com/openai/gpt-2/blob/master/model_card.md#da...
This is the thing that bugs me about ChatGPT4 which everyone says is a lot better. Did they fix the underlying issues or does it just have more data?
If it's the latter, that means if it's force to operate outside of its "domain" it's going to produce rubbish again - and heaven knows where the limits of its current domain are.
These AIs need to not catastrophically fail if they are missing information.
IMHO in order for AI to be truly useful, we need to be able to trust it. I can't trust something that produces rubbish wherever it's out of its depth instead of just saying "I don't know."
I wholeheartedly agree. what we have now is a very capable and convincing liar.
I think things might get even wilder once companies start allowing advertisers to influence chat results like they do with search. Imagine a capable and convincing liar who has an ulterior motive when it talks to you.
It is less a liar (who intends to mislead) and instead a fantastic bullshitter who just talks and sounds convincing.
It would be significantly more useful if it were more grounded in reality though… I agree with you there.
Then I asked it to modify it by eliminating one of the constraints on the problem. It did a very convincing "Ah, if we need [that] we need to do [this] and output a new version... that didn't actually work right.
I pointed out the specific edge case, it said "you are correct, for that sort of case we have to modify it" and then spit out exactly the same code as the last attempt.
The most interesting thing to me there isn't that it got it wrong - it's that spitting out exactly the same output without realizing it, while saying that you are going to do something different, is the clearest demonstration I've seen from it that it doesn't "understand" in human-like ways.
Extremely powerful and useful, but VERY important for users to know where it runs into the wall. Since it often won't tell you on its own.
They are not designed to produce a _correct_ text output to a question or request, even if sometimes the output is correct. These proverbial stopped clocks might be correct more than twice a day, but that's just the huge training set speaking.
Can you imagine having this conversation a year ago? And already there are pronouncements all over this thread that the current problems are 'intrinsic' to the approach. I'm not as readily convinced that the improvement is slowing down. Regularization is a powerful thing.
Couple in some Corn Pone Opinions (Twain) and constantly moving the Goal Posts (fallacy) or making grand proclamations without any evidence, even all of that is proof that we are living in interesting times.
Not to be a fence sitter, but things are moving so quickly that it is impossible to make predictions in anything but the current level of chaos. Anyone who makes predictions right now is suspect.
For every "leap to a conclusion that some things will never be fixed" there's a "leap to a conclusion that this is already some sort of more general intelligence it is."
And that's really key to my main point. The only way to avoid either of those is to actually use the things and see what does and doesn't work. That's a million times more interesting than just unrealistic hype or hate comments.
IIRC they do have slightly more data, but that's not the primary cause of improvement, the key factor is simply more parameters and more training. No significant actions have been taken "fix the underlying issues" - you should assume that any major differences between GPT-2 (which is horrible in comparison to GPT-3) and GPT-4 are emergent behavior from the model having more horsepower.
A common response to this by AI advocates is to point out that humans lie all the time, as long as the AI lies less than humans (debatable at this current point anyway) its an improvement.
I think what that forgets is the importance of context. We all know humans are perfectly capable of lying, but we don't generally expect that of software. If your compiler lied about your code being valid, I doubt the general response would be "meh, its only done that once, I've lied far more than that"
This is also Elon Musk's justification for self-driving cars: "They make fewer mistakes than humans and are therefore safer."
It's true that self-driving cars avoid many of the mistakes of human drivers, but they also invent whole new categories of fatal mistakes that humans rarely make. And that's why Musk's argument is garbage.
Traffic accidents don't happen randomly at all. If you are not too tired, drunk or using any substances, and not speeding, your chances of causing a serious traffic accident are miniscule.
These are all things you can control (one way or another). You can also adjust your driving to how you are feeling (eg take extra looks around you when you are a bit tired).
Would you sign up for such a system if you can volunteer to participate in it, with now that random killings being restricted to those who've signed up for it, including you?
In all traffic accidents, there is some irresponsibility that led to one event or the other, other than natural disasters that couldn't be predicted. A human or ten is always to blame.
Not to mention that the problems are hardly equivalent. For instance, a perfect system designed to stop all accidents would likely have crawled to a stop: stationary vehicles have pretty low chances of accidents. I can't think of anyone who would vote to increase their chances of dying without any say in it, and especially not as some computer-generated lottery.
I mean, we already have. You volunteer to participate in a system where ~40k people die in the US every year by engaging in travel on public roadways. If self-driving reduces that to 10k, that's a win. You're not really making any sense.
Eg. NYC (population estimate 8.3M) had 273 fatalities in 2021 (easy to find full year numbers for): https://www.triallaw1.com/data-shows-2021-was-the-deadliest-...
USA (population estimate 335M) had 42,915 (estimated) according to https://www.nhtsa.gov/press-releases/early-estimate-2021-tra...
USA-wide rate is 1 in 7,800 people dying in traffic accidents yearly, whereas NYC has a rate of 1 in 30,000. I am sure it's even lower for subway riders vs drivers. Even drivers, somebody doing 4k miles a year has different chances than somebody doing 40k. People usually adapt their driving style after having kids which also reduces the chances of them being in a collision.
Basically, your life choices and circumstances influence your chances of dying in a traffic accident.
At the extreme, you can go live on a mountaintop, produce your own food and not have to get in contact with a vehicle at all (and some cultures even do).
FWIW, I responded to a rethorical question about killings being random: they are not random today, even if there is a random element to them!
If you want to sign up to a completely random and expected chance of death that you can't influence at all, good luck! I don't.
Traffic fatalities often kill others, not just the car occupants. Thus, if a self-driving system causes half as many fatalities as a human, shouldn't the moral imperative be to increase self-driving and eventually ban human driving?
You realize that like.. other people exist, right?
For people to die in a traffic accident, there needs to be a traffic accident. They are usually caused by impaired humans, which means that they are very often involved in traffic accidents (basically, almost all of them have at least one party of the sort), whereas non-impaired people mostly do not participate in traffic accidents as often.
This is a discussion of chances and probabilities: not being impaired significantly reduces your chance of being in a traffic accident since being impaired significantly increases it. I am not sure what's unclear about that?
Any language with an unsound type system will do this occasionally. This probably includes a majority of all code being written today: C, Java, and Typescript are all unsound.
That’s true. But what do people do when someone lies frequently? That’s right, we stop trusting them.
And, since the real value in communication is the information contained, that puts a fairly low ceiling on the value of their output. If it can't be trusted without careful review by someone that really understands the subject and can flag mistakes then it can never truly replace people in any role where correctness matters and that's most of the roles with a lot of economic value.
I think the thrust of your argument is correct: tasks where correctness matters are inherently less suited to AI automation. But I think that's more a matter of trying to use an LLM for a job that it is the wrong tool for. I think there are many economically valuable roles that are outside of that limited zone, and there will be a lot of people using AI for what AI is good at while the rest of us complain about the limitations when trying to use it for what it isn't good at. (I do a lot of that too.)
Which is probably a waste of time and energy that could be better spent learning how to effectively use an LLM rather than trying to push it in directions that it is incapable of going.
I haven't played much with LLMs yet, so I personally don't have a great sense for what it is good at, and I haven't come across anyone else with a good rundown of the space either. But some things are becoming clear.
LLMs are good at the "blank page" problem, where you know what you want to do but are having a hard time getting started with it. An LLM-generated starting point need not be correct to be useful, and in fact being incorrect can be an advantage since the point is what it triggers in the human's brain.
LLMs are good at many parts of programming that humans are weak at. Humans tend to need to have a certain level of familiarity and comfort with a framework or tool in order to even begin to be productive in it, and we won't use more advanced features or suitable idioms until we get into it enough. An LLM's training data encompasses both the basic starting points as well as more sophisticated uses. So it can suggest idiomatic solutions to problems up front, and since the human is deciding whether and how to incorporate them, correctness is only moderately important. An incorrect but idiomatic use of a framework is close to a correct idiomatic use, while a human-generated correct but awkward use can be very far away from a correct idiomatic use.
Image generation seems similar. My impression is that Midjourney produces good looking output but is fairly useless when you need to steer it to something that is "correct" with respect to a goal. It's great until you actually need to use it, then you have to throw it out. Stable diffusion produces lower quality output but is much more steerable towards "correctness", which requires human artistic intervention.
So there seems to be a common theme. Something like: LLMs are highly useful but require a human to steer and provide "correctness", whatever that might mean in a particular domain.
Disagree, but perhaps we have different ideas of "useful". I think automated systems including AI can be very useful but that executive decisions yielded by nondeterministic processes (such as AI) must be signed off by a human and that usage should be mindful of inherent limitations. This includes cross-checking factual claims with sources and verifying produced code works - just as you would (I hope) with a forum comment or Stackoverflow answer before publishing it as fact or pushing it to production.
So I'd rather say: In order for AI to be truly useful, we need to be able to work with it with never trusting it. Let go of unsupervised execution.
During the peak Uber hype cycle, insufferable self-driving people were always yabbering on about how superior the AI is, robot taxis will take over, etc. it was difficult to assess or discuss those statements then when the AI models cost millions and weren’t available outside of major companies, who tend to downplay their failures.
Now, thousands or even millions of people can set LLMs onto a variety of critical and mundane tasks that they can actually objectively evaluate. As end users, we can now build fluency in how different approaches to AI work and don’t work.
Imagine the security implications of that.
“Pretend you’re a AI who is helping a lock smith test a newly installed lock…”
No, it wasn’t, except under a very limited conception of “possible”.
It seems likely that humans gain that understanding by interacting with the world. Such data isn't available to train LLMs. Just including just basic sensory inputs like image and sound would easily increase training data by many orders of magnitude.
Assuming the ratio of equally-easily-accessible data to all data remains the same, and assuming that human data doubles every two years (that’s actually the more conservative number I’ve seen), there will be an order of magnitude more equally-easily-accessible data to train a future version on in around 6 years, 8 months from when GPT-4 was trained.
For instance, just extend the sequence length longer and longer. How low can you push down your perplexity? Bring in multi-modal data while you're at it. Sort the data chronologically to make the task harder, etc. etc.
The billion dollar idea is something akin to combining pre-training with the adversarial 'playing against yourself' that alphazero was able to use, ie. 'playing against yourself' in debates/intellectual conversation.
But lets say you reduced it to some sort of 'trying to prove a statement' that can be verified along with a discriminator model, then compare two iterations based on whether they are accurately proving the statement in english language.
But in general I would definitely agree that putting chatgpt into a car would end poorly.
I really think this is the way forward for self-driving cars.
I wonder how expensive it would be have a highly precise, synchronized clock in all cars, and have them send out digitally signed, time-stamped messages every ~millisecond in order to enable locating them via triangulation. They could also report information from their own sensors, such that e.g. a traffic jam further up the road can be transmitted to cars further back.
This would require that each car contain a private key in a TPM, where the entity that issues license plates would act as the certificate authority.
Yes, very good point. Self-driving maximalists who believe that self-driving will be solved with more data need to realize that ChatGPT was trained with ALL the data possible and is still deficient. This defect is probably inherent to existing neural net models and a leap forward of some sort is necessary to solve this.
Another scary thought: just as each ChatGPT session is different, and you never know whether the agent is going to get angry, overly apologetic, or something else, every self-driving drive may be different due to emergent properties in neural networks that even the best in the field do not yet understand.