Scaling Down Deep Learning

Re:Scaling Down Deep Learning by ma2rten on Saturday December 05, 2020 @08:12PM (#25320007)

In general, BERT would be the most common one. RoBERTa is the same model but trained for longer, which turns out to work better. T5 is a larger model, which works better on many tasks but is more expensive.

[ Parent | on HN ]

Re:Scaling Down Deep Learning by ericd on Sunday December 06, 2020 @01:34PM (#25325452)

Thanks for the summary! I'm familiar with BERT, but less so the different variants, so that's quite helpful. I'll take a look at how RoBERTa works.

[ Parent | on HN ]