Re:Markov Chains Are the Original Language Models
by hiddencost on Thursday February 01, 2024 @11:33AM
(#39217767)
Google did it. Amazon did it. Plenty of others did. What do you think they were doing before recurrent neural networks?
[ Parent
| on HN ]
Re:Markov Chains Are the Original Language Models
by ben_w on Thursday February 01, 2024 @01:40PM
(#39219528)
100 million USD?
No.
I'd believe $40 for the energy cost, $120 for very slightly increased wear on their hard drives, and $400 for one engineer's 20% time project for one week.
And that's if it was trained on Google's entire internet cache rather than, say, just a Wikipedia snapshot from 2004, which sounds like the kind of thing that Google might have set as a pre-interview coding challenge.