Learning Theory from First Principles [pdf]

Re:Learning Theory from First Principles [pdf] by throwaway81523 on Saturday March 02, 2024 @04:51PM (#39576086)

This is pretty hard to read. For example, on the first page of chapter 1, it talks about "minimization of quadratic forms" and shows what looks like the formula for linear least squares. Is that right? It doesn't say anything about this. Some more exposition would help.

I do like that there are lots of exercises.

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by guimplen on Saturday March 02, 2024 @05:20PM (#39576275)

I think the text is geared towards people with some mathematical background who want to understand learning theory. Besides it is clearly stated that this chapter is a review (so its assumed that you learned or will learn these things elsewhere).

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by throwaway81523 on Saturday March 02, 2024 @05:42PM (#39576434)

Well I have some math background but that section is brisk and slow at the same time, as it were. Such as how it explains how to find inverses of 2x2 matrices.

This is older but is supposed to be good: https://www.deeplearningbook.org/

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by dawnofdusk on Saturday March 02, 2024 @07:18PM (#39577099)

First principles doesn't mean easy to read unfortunately

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by nerdponx on Saturday March 02, 2024 @05:40PM (#39576420)

The sibling comment is right in that this is clearly not intended for first timers.

But your instincts are correct here. When you write out the objective function for ordinary least squares, it turns out to be a quadratic form. The choice of the word "quadratic" here is not a coincidence: it is the generalization of quadratic functions to matrices. That section covers the vector equivalent of minimizing quadratic functions.

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by garydevenay on Saturday March 02, 2024 @05:44PM (#39576451)

Certainly doesn't seem like first principles...

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by bonoboTP on Sunday March 03, 2024 @10:48AM (#39581581)

"First principles" doesn't mean "introduction". It is to contrast with anecdotal experience / tacit knowledge / empirical best practice approaches.

[ Parent | on HN ]

Re:Learning Theory from First Principles [pdf] by wenc on Saturday March 02, 2024 @06:07PM (#39576638)

Least squares is quadratic.

Quadratic means square terms.

[ Parent | on HN ]