AI training is outpacing Moore’s Law

Re:AI training is outpacing Moore’s Law by oblio on Wednesday December 08, 2021 @02:38PM (#29489383)

I guess the next step is democratizing ML.

Right now we seem to be into a kind of "mainframe phase", where running big models is super expensive.

I wonder what kind of speedup we need for ML to be truly accessible for the average developer. 50x? 100x?

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by ska on Wednesday December 08, 2021 @03:18PM (#29489902)

> speedup we need for ML to be truly accessible for the average developer. 50x? 100x?

The data scale (and access) is often more important than the modelling, and that doesn't have a technological fix, or at least not a purely technological fix.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by fumar on Wednesday December 08, 2021 @03:30PM (#29490087)

So Moore's Law is a constraint for average people as hardware costs a key factor in performance, is that right?

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by ska on Wednesday December 08, 2021 @03:39PM (#29490182)

What I'm saying is the hardware is less of a problem than access to data, for most problems that would get you out of what you can reasonably do on a consumer accessible (cost) machine.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by jsharf on Wednesday December 08, 2021 @05:05PM (#29491173)

Unless we have widely available pre-trained models with generic knowledge that is easily transferable. Such a thing is obviously difficult, but it isn't impossible if we're talking about the future.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by hervature on Wednesday December 08, 2021 @02:40PM (#29489411)

Define "accessible". Very modest hardware (sub $100 and certainly sub $1000) can train random forests very quickly. This is going to be close to SOTA on basically any classification task.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by acover on Wednesday December 08, 2021 @07:55PM (#29492576)

Random forests weren't even state of the art in 2010.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by hervature on Friday December 10, 2021 @12:02AM (#29506589)

Are random forests new? no. SOTA in what? Image classification, obviously no. But they are still going to perform very well in the vast majority of business problems in classification.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by minimaxir on Wednesday December 08, 2021 @03:26PM (#29490030)

The trick for ML accessibility is minor finetuning the existing models that have been trained on these super training models so that users can get similar performance, and lately companies have been more open about releasing models open source.

[ Parent | on HN ]

Re:AI training is outpacing Moore’s Law by mistrial9 on Wednesday December 08, 2021 @06:49PM (#29492132)

Google is all over this -- there are at least two science-fiction level videos on YouTube of actual field experiments using ML, each with a very non-USA/third world person narrating.. the view count is in the low four digits both times

one of them is related to a paper in Nature this year on "democratizing ML" .. the MOSAIK project iir

edit- hilariously, Google Search wont let me look for MOSAIK instead correcting the spelling.. and when forced, many similarly named projects that are not it, are ahead.. the one referenced here is

    A generalizable and accessible approach to machine learning with global satellite imagery

[ Parent | on HN ]