Re:Yann LeCun on GPT-3
by ivalm on Tuesday October 27, 2020 @03:14PM
(#24910750)
In general GPT3 is not SotA on (any?) classification task, did you just not have enough data to fine tune a discriminative transformer model? Inference should be cheaper with a smaller transformer/also less lock-in.
[ Parent
| on HN ]
Re:Yann LeCun on GPT-3
by neural_thing on Tuesday October 27, 2020 @03:22PM
(#24910827)
I can't go into too much detail here about why we couldn't do that, but one aspect that we found VERY useful is that GPT-3 could draw on real world knowledge not present in the dataset to enhance the results.