How to train large deep learning models as a startup

Re:How to train large deep learning models as a startup by aabaker99 on Thursday October 07, 2021 @06:59PM (#28793228)

1. Gradient descent almost always finds a non optimum local min (it is not guaranteed to find a global min).

[ Parent | on HN ]

Re:How to train large deep learning models as a startup by agnosticmantis on Thursday October 07, 2021 @07:06PM (#28793288)

Isn’t the current best practice to train highly over-parametrized models to zero training error? That’d be a global optima, no?

Unless we’re talking about the optima of test error.

[ Parent | on HN ]

Re:How to train large deep learning models as a startup by aabaker99 on Thursday October 07, 2021 @07:35PM (#28793561)

If you find a zero in a non negative function, I would call that a global minima, yes.

[ Parent | on HN ]

Re:How to train large deep learning models as a startup by aledalgrande on Thursday October 07, 2021 @08:00PM (#28793756)

Yeah but depending on the data you might have even worse results, selecting the right subset to be representative is really important.

[ Parent | on HN ]

Re:How to train large deep learning models as a startup by jhgb on Thursday October 07, 2021 @11:43PM (#28795510)

Would a random sample be representative? Statistically this seems to be the case for any large N. In fact it's not clear to me that any other sample would be more representative.

[ Parent | on HN ]

Re:How to train large deep learning models as a startup by aledalgrande on Friday October 08, 2021 @01:03PM (#28801456)

Many public datasets have skewed classes so if you take a random approach you're not gonna have a good result. And N might not be big enough anyway.

[ Parent | on HN ]