raptortech

Abstract: Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.

1. A plot over time

2. Could you adjust the income for inflation, so when you click back 1yr your income goes down accordingly?

I would love to know what "watching videos" means here. There's a big difference between educational YouTube (Kurzgesagt, Physics Girl, Vertitasium, etc) and TV.

Why use a learned policy instead of a classical controller, you ask? The classical controller is only as good as the dynamics model. The learned policy, when trained with domain randomization, is robust to error in the dynamics model and external disturbances.

I'm excited about learned search heuristics in theorem space.

But I think that with deep learning, there is now a real opportunity to make a kickass visualizer. Take a GAN and set the latent vector according to the DFT of the audio stream -- it would be super trippy. You could even use WaveNet and CLIP to make the visuals match the lyrics.

If we make the (strong) assumption that sleep is primarily a cognitive repair period, then maybe this isn't too surprising? It would be interesting to do the same study, but measuring cognitive ability instead of risk of death.

"Power generation from solar PV is estimated to have increased by 22% in 2019, to 720 TWh" [1]

"Bitcoin miners are expected to consume roughly 130 Terawatt-hours of energy (TWh)" [2]

[1] https://www.iea.org/reports/solar-pv [2] https://techcrunch.com/2021/03/21/the-debate-about-cryptocur...

Agreed. That being the case, do you really want to read 20-200 pages of introduction and as much conclusion with a relatively low channel capacity? That simply cannot compare to watching a high quality educational youtube video at double speed.

"You want to consume content that took high amounts of effort to create because it gives you the most bang for your buck."

Youtube is free; books are not.

No, the real reason books are great is because they are slow enough that they give you the time to think about what you are reading. Not because they are highest value.

-1 for formatting. Can we at least get a monospace font here?

1. learning a policy that imitates your own behavior on prior experience, which is a trivial supervised learning problem

2. learning how to weight the importance of prior experiences (learning a data distribution), for which the authors have derived a lower bound

Given a pool of experience, this seems like a fantastic off-policy method to optimize arbitrary reward functions. The main shortcomings I see with this method is that it still does not lead to any significant insights into how to collect new data online, which is a major open problem in RL.