https://arxiv.org/abs/1706.03762
Why was this revolutionary though?
Attention is all you need paper just proposed an AR model that didn’t have to be trained step by step. The scaling happened later in BERT and GPT and OpenAI’s scaling work
* "Attention is all you need" introduced positional encoding which allows you to keep context of the word, allowing for more complex translation (and thus generative/chatgpt like tasks?) because words now have context relative to each other. Contrast this with "bag of words" models that only tells you whether the word is present or not.
* I don't quite understand why but transformers (which "AiaYN" introduced) can be made fully parallel, compared with the RNN/LSTM networks which has to be serial per token. Fully parallel allows for GPU optimization, which means you can take advantage of Moore's law for training.
I'm always a bit suspicious when people claim a breakthrough of this sort. There's no doubt that better algorithms give better results but how much is due to just faster computers, cheaper compute, memory, etc.
That said, this doesn't really seem all that comparable. The article points out very fundamental properties of all the diverse current approaches: They are tightly data constrained. You either need to cheap simulation or massive real world data. That's not an arcane technical point.
> conveniently sidestep[s] the possibility of the development of new tools and techniques.
I don't believe they did this at all. Here is their summary of their third point:
> To automatically construct a good dataset, we require an actionable understanding of which datapoints are important for learning. This turns out to be incredibly difficult. The field has, thus far, completely failed to make progress on this problem, despite expending significant effort. Cracking it would be a field-changing breakthrough, comparable to transitioning from alchemy to chemistry.
That does not read to me as conveniently sidestepping the possibility of new tools. Rather, it is acknowledges that to overcome the problem we NEED a new tool, aka a breakthrough.
I also disagree with the framing of the third section as:
> And that entire section basically boils down to "I personally believe this is hard and will take a long time."
They provide both theoretical and empirical evidence of their claim. I find it was well argued, and I'm inclined to agree. Every scenario I can think of with a runaway superintelligence requires a way to automatically improve datasets as part of the learning process.
"Also, they aren’t getting more-solved over time: we’ve made little-to-no progress on any problem of this sort in the last decade, certainly not the reliable improvements of the sort we’ve seen from supervised learning. This indicates that a breakthrough is needed — and that it is unlikely to be close."
Past lack-of-progress is not an indicator of future lack-of-progress. "unlikely to be close" is unknowable, and is just the author's gut feeling.
Past lack-of-progress is not proof of future lack-of-progress. But it's most definitely an indicator.
> "unlikely to be close" is unknowable, and is just the author's gut feeling.
Again I'm sorry but I have to disagree with you here. The very next paragraph reads:
> Something that would change my mind on this is if I saw real progress on any problem that is as hard as understanding generalization, e.g. if we were able to train large networks without adversarial examples.
Basically, the author has identified a class of problem. They are claiming that little to no progress has been made on any problem in that class. So it's not just that no progress has been made on this specific problem, but that we are stuck on all problems of this type. Thinking of it in that way, I do not find it unreasonable to say we are "unlikely to be close" to a solution.
If you have a CS background, I'll make an analogy to computability classes: It's like the author is saying this is an NP hard problem. We have made no progress on any NP hard problem. I think it's reasonable to say that we are unlikely to be close to solving a particular NP hard problem because we've made no progress on any NP hard problem.
You're free to disagree with the author's premises. E.g. "this problem is not like those other problems" or "we actually have made progress on those other problems". But I think that the conclusions are reasonable given the premises.
The only reason we have things like ChatGPT today is because of the surprise development of the transformer. Nobody understands how intelligence fundamentally works, so we literally have no ability to predict when the next groundbreaking architectural development will occur.
And self-improvement concerns are mostly orthogonal to the main risk of AI: the development of AGI. Even a mediocre AGI that is only "decent" at tasks, can be weaponized as a collective superintelligence.