Re:Show HN: Convert any screenshot into clean HTML code usin...
by cornel_io on Thursday November 16, 2023 @06:08AM
(#38288138)
Specifically, the "universal function approximate" thing means no more and no less than the relatively trivial fact that if you draw a bunch of straight line segments you can approximate any (1D, suitably well-behaved) function as closely as you want by making the lines really short. Translating that to N dimensions and casting it into exactly the form that applies to neural networks and then making the proof solid isn't even that tough, it's mostly trivial once you write down the right definitions.
Re:Show HN: Convert any screenshot into clean HTML code usin...
by yu3zhou4 on Thursday November 16, 2023 @06:11AM
(#38288166)
Specifically for neural networks, is there any alternative for backpropagation and gradient descent which guarantee finding the global minimum?
Re:Show HN: Convert any screenshot into clean HTML code usin...
by b3kart on Thursday November 16, 2023 @06:41AM
(#38288352)
Unlikely given the dimensionality and complexity of the search space. Besides, we probably don’t even care about the global minimum: the loss we’re optimising is a proxy for what we really care about (performance on unseen data). Counter-example: a model that perfectly memorises the training data can be globally optimal (ignoring regularization), but is not very useful.
The learning algorithm used: Backpropagation with Stochastic Gradient Descent is not the universal learner. It's not guaranteed to find the global m