wish5031

Have you read Goodfellow / Bengio / Courville’s deep learning book? The later chapters go in more depth than most other resources I’ve found.

It’s questionable how “immature” ML really is. Most methods that get used were initially designed 50+ years ago, with various improvements over time. E.g., neural networks were invented in the 1950s, backprop was introduced in the 80s, architectures like LSTM and CNN in the 90s, etc.

The only thing that’s really new is the amount of computational power at our hands. That has allowed us to shift from relatively simpler methods to more powerful but opaque methods like NNs. They just don’t lend themselves to easy analysis because it’s a lot harder to explain why inputs to these ML systems map to their respective outputs. Hence, attempts at drawing the connection between inputs and outputs become more speculative.

At the beginning the authors claim they're not just making a restatement of the Stone-Weierstrass theorem (any continuous function on a compact set can be approximated arbitrarily well by a polynomial), but reading through, I'm not sure what they're proving besides that. In fact, on pages 6-7 they simply appeal to that theorem before stating "NNs can loosely be viewed as a form of polynomial regression". Most of the rest of the paper is comparing polyreg and NNs on various datasets. Sec. 9 doesn't have anything doesn't have anything especially novel in it. For instance, in 9.3 they say they'll explore the causes of overfitting in the context of their "NN <-> PR" principle, but never actually do so...

Polynomial regression is nice because it's a little easier to interpret, and it's also a convex problem with a single, global minimum. OTOH you have to design features yourself, otherwise for high-dimensional problems polyreg quickly requires way too much memory and compute to solve in a reasonable amount of time.

This paper might have been more interesting if it had somehow connected neural nets to those facts, or if it had shown how insight on the way in which neural nets work can be used to improve polyreg. But (admittedly, from a brief reading) I don't anything like that here.

If this interests you, a related concept with similar applications as HE is functional encryption: https://en.m.wikipedia.org/wiki/Functional_encryption

Nice! I like that the author wrote the code by hand rather than leaning on some framework. It makes it a lot easier to connect the math to the code. :)

As a meta-comment on these "Introduction to _____ neural network" articles (not just this one), I wish people would spend more time talking about when their neural net isn't the right tool for the job. SVMs, kNN, even basic regression techniques aren't any less effective than they were 20 years ago. They're easier to interpret and debug, require many fewer parameters, and potentially (you may need to apply some tricks here or there) faster at both training and evaluation time.

I used to play a lot, and hit 1900 (USCF) before I ran out of time to keep playing competitively. So I'm not a pro by any means, but I might know enough to help you get started.

The best way to increase your rating is to find a good chess coach. Otherwise, off the top of my head, my suggestions would be (in order of importance):

1. Learn an opening really well. Ideally, find one opening for white that doesn't generally have many weird transpositions, e.g. the London system. Also, find a couple of solid defenses for black in response to 1. e4 and 1. d4, e.g. the French and the Slav defenses.

2. Study the most basic endgames -- queen + king vs king, rook + king vs king, etc. I'd suggest [0] as a resource.

3. Practice a lot of tactics. You can do this through books (I'm partial to [1] for beginners and [2] when you reach 1600 or so) or through online services like lichess.org.

4. You can probably hit 1500-1600 fairly easily with 1-3. At this point I'd start learning the basics of strategy/positional play, e.g. static vs dynamic advantages. I think [3] is a pretty good resource for this.

5. Study some of the more advanced endgames in [0].

6. Start looking at some more complex openings and go a bit deeper into positional play. IMO the most important thing in the long run is just to find an opening and style of play that you really like.

Hope that helps!

[0] https://www.amazon.com/Endgame-Challenge-John-Hall/dp/188067... [1] https://www.amazon.com/Combination-Challenge-Lou-Hays/dp/188... [2] https://www.amazon.com/Sharpen-Your-Tactics-Sacrifices-Combi... [3] https://www.amazon.com/Elements-Positional-Evaluation-Dan-He...

Hey there -- I'm a co-founder of a small company working in exactly this space. If at all possible, it'd be great for us to get in touch!

But there actually is a huge amount of theory behind that problem. You can exactly derive the method that finds the best line. You can get error bounds on each of your coefficients and confidence intervals for them. You can alter the strength of your assumptions (e.g. about distribution of errors, homoskedacity, and so on) and see how it affects your model. You can add L1 or L2 regularization, both of which also have solid theoretical grounding. And so on.

All of these things help make your model more robust and give you greater confidence in it, which will be important if we want to put ML in, say, healthcare or defense. But you don’t get as much of this theory with more complex ML models, and certainly not with neural nets. Good luck trying to get a confidence interval for the optimal value of a weight in your net, much less interpreting it.

anecdotally, the adapter Apple provided me broke after about three weeks, and it seems like most of my family/friends are in similar situations

Hm, I see. Thanks!

Talking about LWE, the author says:

> A major problem with this system is that it has very large keys. To encrypt just one bit of information requires public keys with size n^2 in the security parameter.

Can't this be solved by e.g. forcing some kind of random sparsity structure on the matrix and then compressing with a format like CSC or CSR? (probably just not understanding LWE completely)

I have a few friends who shared a poorly Photoshopped picture of Hilary Clinton shaking hands with bin Laden. That helped reinforce some of their nutty beliefs during the last election cycle.

An example of how dangerous video can be is the faked Planned Parenthood video from a couple years back. That actually led a guy to shoot some people in a PP clinic.

That seems like a plausible argument, but there’s a huge portion of the population that’ll willingly believe any realistic-looking video they see, regardless of the evidence. That could have fatal consequences, e.g. as happened with the Planned Parenthood video in 2015. [0]

[0] https://en.m.wikipedia.org/wiki/Planned_Parenthood_2015_unde...

As others have mentioned, Tao has fantastic resources, and his blog might be worth looking at. Rudin is a classic. I studied real analysis through “Advanced Calculus” by Fitzpatrick which has some great material.

For more advanced analysis (esp. functional analysis) I would look at Kreyszig or Hunter & Nachtergaele.

The best way to prepare imo is to just do proofs between now and the start of the course. Try to find practice proof problems online and see if you can do them or find an entry-level book on discrete math. Problem-Solving Strategies by Engel is a good but slightly more advanced book for a beginner.