Re:Deep learning job postings have collapsed in the past six...
by p1esk on Monday August 31, 2020 @02:00PM
(#24334136)
Will GPT-4 be any different?
That's the point - it does not need to be different. If it demonstrates similar improvement to what we saw with GPT-1 --> GPT-2 --> GPT-3, then it will be enough to actually start using it. It's like the progression MNIST --> CIFAR-10 --> ImageNet --> the point where object recognition is good enough for real world applications.
But in addition to making it bigger, we can also make it better: smarter attention, external data queries, better word encoding, better data quality, more than one data type as input, etc. There's plenty of room for improvement.
Re:Deep learning job postings have collapsed in the past six...
by disgruntledphd2 on Monday August 31, 2020 @12:38PM
(#24333132)
No, almost all the progress is driven by bigger GPUs and datasets.
To be fair, things like CNN's and BERT were definitely massive improvements, but a lot of modern AI is just throwing compute at problems and seeing what sticks.
It seems like the current approaches will always fall short of our loftier AI aspirations, but we’re reaching a level of mimicry where we can start to ask, “Does it matter for this task?”