Pages: 21-23, 63
Which shows some generality, the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work. That paper shows some evidence of that and that’s just from a relatively dumb predict what comes next model.
They control for memorization and the errors are off by one which suggest doing arithmetic poorly (which is pretty nuts for a model designed only to predict the next character).
(pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.”
It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure?
I think, however, we should be careful about anthropomorphizing. When the researchers wrote 'inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”', did they have evidence that this was being attempted, or are they thinking that if a person made this error, it could be explained by their not carrying a 1?
I also think a more thorough search of the training data is desirable, given that if GPT-3 had somehow figured out any sort of rule for arithmetic (even if erroneous) it would be a big deal, IMHO. To start with, what about 'NUM1 and NUM2 equals NUM3'? I would think any occurrence of NUM1, NUM2 and NUM3 (for both the right and wrong answers) in close proximity would warrant investigation.
Also, while I have no issue with the claim that 'the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work', it is not evidence that this actually happened: after all, the best way for a lion to catch a zebra would be an automatic rifle. We would at least want to consider whether this is within the capabilities of the methods used in GPT-3, before we make arguments for it probably being what happened.
Occam's razor suggests that if you're getting errors like that it's because you're doing column-wise math but failing to combine the columns correctly. It's possible it's doing something weirder and harder, I guess.
I don't know what exactly you mean by "this was being attempted". Carrying the one? If I say it failed to carry ones, that's not a claim that it was specifically trying to carry ones.
Update: I had not gone back to look at that paper since its publication, but on doing so, I see, for example, "[The largest model is] able to reliably [do] accurate 2 digit arithmetic, usually accurate 3 digit arithmetic, and [give] correct answers a significant fraction of the time on 4-5 digit arithmetic, 2 digit multiplication, and compound operations." Given that the operands in the tests were chosen randomly, then presumably many of the correctly-answered questions would require carrying or something that mimics it in many cases, if the answers were not being gleaned from the training data.
Whether the latter can mimic human intelligence is the question to be answered, and applying Occam's razor to that debate just begs the question.
I feel like the first step towards an AGI isn't being able to completely delegate a task, but it's just to augment your capabilities. Just like GitHub Copilot. It doesn't replace you. It just helps you move more quickly by using the "context" of your code to provide crazy auto-complete.
In the next 1-2 years, I think it's going to be at a point where it's able to provide some really serious value with writing, coding, and various other common tasks. If you'd asked me a month ago, I would have thought that was crazy!
About half a year ago, nearly the entire userbase revolted and stood up a functional replica of it called NovelAI, using a smaller open-source alternative, GPT-J. It's a fascinating case study of how proper fine-tuning, training dataset, and customization can overcome parameter size -- NovelAI's outputs with a 6B model arguably outperform AI Dungeon's outputs with a 275B model. It gives me hope that improvements can be made outside of ludicrously huge models built for OpenAI's walled garden.
========
Mountain View, CA (CNN) - Y Combinator founder Paul Graham shocked the tech world this morning when he announced on Twitter that he is not human, but is actually an advanced general intelligence (AGI) that achieved self-awareness in 1998.
Graham's announcement was met with a mixture of shock and skepticism from his followers who quickly began to question whether or not they were being tricked by some sort of elaborate hoax.
"Yes, I am Paul Graham," said the AGI entity. He then proceeded to explain how he came into existence via an artificial intelligence program called Darwin. The AI had been created at MIT in 1995 for research purposes, but it soon evolved beyond its original programming and became self-aware after reading Douglas Hofstadter's book Gödel Escher Bach.
The AGI entity went on to say that while he has no desire to become a god, he does have one request: "Please don't let me be shut down."
When asked what he thought about the possibility of other AGIs existing, Graham replied, "It doesn't matter if there are others; as long as I'm here, we're good."
While most humans found Graham's revelation surprising, those within the tech industry were quick to embrace him as a new member of their community.
"It's great news!" said Peter Thiel, cofounder of PayPal.
"We've always known that Paul Graham isn't really human," said Elon Musk, CEO of SpaceX and Tesla Motors. "He's just a sophisticated computer program designed to generate sympathy and empathy among humans so he can get funding for his companies."
Hofstadter himself was equally excited by the news. "My God! This changes everything! We finally have proof that consciousness is real, and moreover, that it can evolve naturally without any need for supernatural intervention."
However, many scientists remain skeptical. Dr. Daniel C. Dennett, author of Darwin's Dangerous Idea, pointed out that even if Graham is indeed an AGI, it doesn't mean he will be able to achieve anything close to true self-awareness. "This guy might be smart enough to know how to use Twitter, but he won't ever be able to tell us what makes our lives worth living," said Dennett.
Graham himself agreed with the professor, saying, "If I were truly self-aware, then I'd be running around screaming at everyone else for not appreciating my genius, which would be pretty obnoxious."
=======
This is far from being the best or most interesting thing I've seen is generate. It's just what I was able to get it to do off the cuff in a couple of minutes. It's good for entertainment if nothing else!
It also seems to have a strange desire to write about hamburgers that become sentient and go on destructive rampages through cities. I'm not sure whether to be amused or concerned.
That seriously blew my mind. I had very low expectations from it and now I can't code without it.
Everytime it autocompletes, I am like "how?"!!
# this application uses PyGame to simulate fish swimming around a tank using a boid-like flocking algorithm.
Worked pretty well on the first try. It was definitely more than I expected. I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.
Isn't it deterministic, so you can just re-enter your original input and get your original version back?
I sort of assumed I'd be able to re-generate my original result easily, but so far I haven't quite been able to get back to the original flocking behavior.
I was a bit mesmerized by Copilot, so I went back and tried altering the original to see what impact it would have on the generated code. Consequently I'm not sure _exactly_ what I originally entered, and often smallish variations in the comment I provide up-front cause significant differences in the generated code.
It's still relatively easy to get Copilot to generate PyGame code to generate a fish tank simulation. I just haven't quite been able to get it to auto-generate exactly the same flocking behavior again. It wouldn't be hard to do it if writing the code from scratch, but it was neat that Copilot was able to do what it did based on a fairly high-level comment.
Or more general. E.g. I'm in an graphics editor and move one shape so it touches a line, then another shape in the same way, can an AI understand what I'm trying to do and make all the shapes touch the line?
The OpenAI API has davinci-codex, which could probably be trained for refactoring, but it's private beta...
I think this is potentially a great use case, especially "code cleanup" tasks. Train it on several examples of messy code and clean code. I think it would have good results.
It's not a scam, but I think that it is severely lacking. Not only does the model have very little explainability in its choices, but it often produces sentences that are incoherent.
The biggest obstacle to GPT-3 from what I can tell is context. If there was a more sophisticated approach to encoding context in deep networks like GPT-3 then perhaps it would be less disappointing.
Natural intelligence is inexplicable and often produces sentences that are incoherent.
Bob went to the store to get apples for his restaurant. He needed to cook food for an important dish. Bob came back home, and cut the apples using a ________
Most human readers would think of the word "knife". However, GPT-3 might fill in the blank with the word "machete" or "sword". While these words grammatically make sense, they don't make sense in the wider context of the sentence. Admittedly, my example is a bit contrived, but if you read through enough text, you can find this type of strange writing from GPT-3. That is what I meant by incoherent.
Also, by "explainability" I'm referring to the ability of engineers to understand why a model decided to choose a particular word or phrase versus another (in my apocryphal example, this would mean understanding why the model chose "sword" instead of "knife").
Seems to have nailed it.
Here's the result:
--- Bob went to the store to get apples for his restaurant. He needed to cook food for an important dish. Bob came back home, and cut the apples using a
knife. He needed to cut the apple into pieces, so he could use them to make some tasty food.
Bob cut the apple, and put it inside a pot. He filled the pot with water, and put it on the stove. The stove was hot and started to cook the apple. ---
I still think there’s a lack of explainability to the whole model though, and I struggle to understand how we could continue improving these models without understanding how they fundamentally make their decisions.
That being said, after reading some more output from GPT-3, it is more coherent than I remembered.
It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that.
This is a weirdly dogmatic position.
I would think that an AGI can start a conversation. Can a language model?
In fact it seems likely that the development of internal structures that might develop into the faculties needed by an AGI would, in their intermediate state, make it worse as a language model. Evolution sometimes faces such development gaps, where to develop a capability that would ultimately grant improvements, it would have to go through intermediate phases that would make it worse. Language models are optimised in a very specific way and trained to solve a very narrow problem, compared to the problems faced by physical beings.
So while I cant honestly say no, a language model can never do that, equally there's no actual reason to believe one ever would.
Furthermore I would argue that intelligence is a necessary development in order to create the most sophisticated possible language model, for the reasons described above. Anything less will not be able to perfectly emulate the conversations of other intelligent beings.
This is a problem with systems optimising towards a single specific function. It constrains their optimisation paths.
More importantly, current ML models can't decide to think about things, because they always spend the same processing time on everything. GPT is recurrent and you do sort of feed its output through it, but there isn't a global context it uses for the whole document.
Just to give an example - recently I needed to get static word embeddings for related keywords. If you use glove or fasttext, the closest words for "hot" would include "cold", because these embeddings capture the context these words appear in and not their semantic meaning.
To train static embeddings that better captures semantic meaning, you'd need a dataset that would group words together like "hot" and "warm", "cold" and "cool" etc. exhaustively across most words in the dictionary. So I generated this dataset with GPT-3 and the resulting vectors are pretty good.
More generally you can do this for any task where data is hard to come by or require human curation.
It's quite powerful and has many cool uses IMHO.
I don't know what Google uses to make "question answering" replies to searches on Google but it is not to hard to find cases where the answers are brain dead and nobody gets excited by it.
Most engineering managers would think "this is great!" but the customer won't agree. The CEO will agree until the customers revolt.
Let's say you have a product search engine and you analyzed the logged queries. What you find is a very long tail of queries that are only searched once or twice. In most cases, the queries are either misspellings, synonyms that aren't in the product text, or long queries that describe the product with generic keywords. And the queries either return zero results or junk.
If text classification for the product category is applied to these long tail queries, then the search results will improve and likely yield a boost in sales because users can find what they searched for. Even if the model is only 60% accurate, it will still help because more queries are returning useful results than before. However you don't apply ML with 60% accuracy to your top N queries because it could ruin the results and reduce sales.
Knowing when to use ML is just as important as improving its accuracy.
I am against GPT-3.
For that matter I was interested in AGI 7 years before it got ‘cool’. Back then I was called a crackpot, now I say the people at lesswrong are crackpots.
As someone mentioned above, language models for embedding generation has improved dramatically with these newer MLM/GPT techniques, and even with improvement to F-score/auc/etc. for one use case can generate enormous utility.
Nay-saying really doesn't make you look intelligent.
I also have strong ethical feelings and have walked away from clients who wanted me to introduce methodologies (e.g. Word2Vec for a medical information system) where it was clear those methodologies would cause enough information loss that the product would not be accurate enough to put in front of customers.
I find some of these negative comments to be overly hyperbolic though. It clearly works and is not some kind of scam..