News for hackers. Stuff that matters.

A comment on M.I.T. Plans College for Artificial Intelligence, Backed by $1B. Parent comment

Re:M.I.T. Plans College for Artificial Intelligence, Backed ... by mr_overalls on Monday October 15, 2018 @11:32AM (#18220513)
How important is optimization? Like a lot of engineers I know, I took a few required courses in undergrad on linear algebra and stats. But I've never studied optimization. And I see it come up all the time on lists of central theory for ML & AI. . .

Is there a classic textbook on the subject? Are there any free online courses that are considered good?

https://www.coursera.org/courses?query=optimization

[ Parent | on HN ]
Re:M.I.T. Plans College for Artificial Intelligence, Backed ... by snakeboy on Monday October 15, 2018 @11:35AM (#18220546)
Nocedal & Wright [1] is generally considered the definitive text for reference, at least. Very clear and complete from what I recall.

[1] https://www.springer.com/us/book/9780387303031

[ Parent | on HN ]
Re:M.I.T. Plans College for Artificial Intelligence, Backed ... by Puer on Monday October 15, 2018 @11:36AM (#18220554)
Convex Optimization by Lieven Vandenberghe and Stephen P. Boyd was the one my professors always recommended and I think it's excellent.
[ Parent | on HN ]
Re:M.I.T. Plans College for Artificial Intelligence, Backed ... by Eridrus on Monday October 15, 2018 @01:43PM (#18221828)
Optimization as a tool is important and widely used, but... almost everything grabbing headlines uses some form of SGD + Momentum. Very little of the actual progress comes from better optimization.
[ Parent | on HN ]
Re:M.I.T. Plans College for Artificial Intelligence, Backed ... by tnecniv on Monday October 15, 2018 @07:05PM (#18224803)
Optimization can be pretty useful. Most stats / ML problems are posed as minimizing a function subject to some constraints.

Depending on your problem, you might be able to exploit special structures to solve problems faster than just doing gradient descent. If you know linear algebra and stats, you'll be fine getting through an optimization book.

Boyd's book is canonical at this point, but might be hard to get through. Before you get to actually optimizing anything, you need to make your way through some chapters on convex analysis with little application.

[ Parent | on HN ]