I think it's mostly about having deeper pockets than being clever or original when it comes to A.I. (though these guys just got $15m fwiw).
But this is one thing that worries me about A.I speaking as an indiehacker.
Being scrappy won't soon cut it anymore. Right now you can bootstrap a solid product for $15K, at least a MVP. Soon with the amount of GPU and other resources required to train these models you'll need a min $1M budget and no amount of jugaad will be able to cut it.
Re:Next-generation AI models
by BrandonJung on Wednesday June 22, 2022 @07:50AM
(#31834810)
Cost wise the biggest spend is building the large model (CODEX for Copilot). While CODEX is close sourced there are a number of other good open source transformer models from SFDC (T5), AI21, Meta that are the foundational models we use. Going forward we will continue to partner with the community to enable these large models to reach developers in code-time.
Just out of curiosity since you are so knowledgeable in this area.. what would be like a very approx ballpark figure to develop something like codex?
Re:Next-generation AI models
by BrandonJung on Thursday June 23, 2022 @10:34AM
(#31849667)
Ballpark guess is that to build CODEX it would be ~$10M.
What they're doing is impactful, impressive - and so eclipsed by an unreasonably well-resourced competitor that they don't even get given much credit.
"Not as good as CoPilot" really doesn't say enough about something that's still very impressive.
Personally I thought it was cool but not game-changing, and moved on when I got into CoPilot. I can't comment on the next-gen models.