joaorico

Kafka [1] on which types of book to read:

"I believe one should only read those books which bite and sting. If the book we are reading does not wake us up with a blow to the head, then why read the book? To make us happy, as you write? My God, we would be just as happy if we had no books, and those books that make us happy, we could write ourselves if necessary. But we need the books that affect us like a disaster, that hurts us deeply, like the death of someone we loved more than ourselves, like if we were being driven into forests, away from all people, like a suicide, a book must be the axe for the frozen sea inside us." [2]

[1] Brief an Oskar Pollak, 27. Januar 1904. , https://homepage.univie.ac.at/werner.haas/1904/br04-003.htm

[2] Literal translation by ChatGPT. Original:

"Ich glaube, man sollte überhaupt nur solche Bücher lesen, die einen beißen und stechen. Wenn das Buch, das wir lesen, uns nicht mit einem Faustschlag auf den Schädel weckt, wozu lesen wir dann das Buch? Damit es uns glücklich macht, wie Du schreibst? Mein Gott, glücklich wären wir eben auch, wenn wir keine Bücher hätten, und solche Bücher, die uns glücklich machen, könnten wir zur Not selber schreiben. Wir brauchen aber die Bücher, die auf uns wirken wie ein Unglück, das uns sehr schmerzt, wie der Tod eines, den wir lieber hatten als uns, wie wenn wir in Wälder vorstoßen würden, von allen Menschen weg, wie ein Selbstmord, ein Buch muß die Axt sein für das gefrorene Meer in uns."

I suppose it's quite off-topic, but some weeks ago I read a small book by Mary Gaitskill, the writer of the piece.

It's called "Lost Cat".

I highly recommend it. Ironically, it might be an approximate opposite of Pale Fire. It's very short, with simple yet beautiful prose, filled with intense, raw emotions.

For anyone diving into Ulysses, I highly recommend checking out The Joyce Project [1].

It's filled with interactive notes that are very useful for understanding the linguistic and cultural references.

Here's my reading method that I found effective:

  1. Read a section on paper.
  2. Go through the same section on the site.
  3. (Re-)read on paper.

I toggled between 1-2-3, 1-2, or 2-3 depending on my mood, and it worked really well.

[1] https://www.joyceproject.com/

Incidentally, if you’re looking to start reading in French, there is hardly a better book in terms of (impact on literature) times (simple, accessible writing). It’s also a short book.

Regarding the literary merit of Camus, Nabokov had this to say [1]:

”I happen to find second-rate and ephemeral the works of a number of puffed-up writers—such as Camus, Lorca, Kazantzakis, D. H. Lawrence, Thomas Mann, Thomas Wolfe, and literally hundreds of other “great” second-raters.”

“Brecht, Faulkner, Camus, many others, mean absolutely nothing to me, and I must fight a suspicion of conspiracy against my brain when I see blandly accepted as “great literature” by critics and fellow authors Lady Chatterley’s copulations or the pretentious nonsense of Mr. Pound, that total fake.”

“Incidentally, I frequently hear the distant whining of people who complain in print that I dislike the writers whom they venerate such as Faulkner, Mann, Camus, Dreiser, and of course Dostoevski.”

“It is a shame that he [Franz Hellens] is read less than that awful Monsieur Camus and even more awful Monsieur Sartre.”

[1] Strong Opinions

Opportunity costs. The real debate has been whether it makes sense for string theory (whatever the prevailing definition is) to dominate funding for theoretical research of the "bridge". There are alternatives besides strings for the bridge, and there should be even more, in theory...

Simpson’s paradox should be taken into account.

If you group the population into only 2 groups: all of the vaccinated and all the unvaccinated, regardless of age; then the vaccinated had a higher death toll.

But age is a hidden factor. The older have more risks and are more vaccinated.

If you group by vaccination AND by age bracket, the opposite happens. For example, the 60 to 65 vaccinated have a lower death rate than the 60 to 65 unvaccinated.

Alain Connes, Fields medalist, talks about going on walks while reading math books in a particular way (and on how a mathematician works and should read a book) [0]:

"To understand any subject, above all, a mathematician SHOULD NOT pick up a book and read it.

It is the worst error!

No, a mathematician needs to look in a book, and to read it backwards. Then, he sees the statement of a theorem. And, well, he goes for a walk. And, above all, he does not look at the book.

He says, "How the hell could I prove this?"

He goes for his walk, he takes two hours ... He comes back and he has thought about how he would have proved it. He looks at the book. The proof is 10 pages long. 99% of the proof, pff, doesn't matter.

Tak!, here's the idea!

But this idea, on paper, it looks the same as everything else that is written. But there is a place, where this little thing is written, that will immediately translate in his brain through a complete change of mental image that will make the proof.

So, this is how we operate. Well, at least some of us. Math is not learned in a book, it cannot be read from a book. There is something active about it, tremendously active.

[...]

It's a personal, individual work."

[0] https://www.youtube.com/watch?v=9qlqVEUgdgo

In case you're interested in learning about graph deep learning, and are familiar with standard DL, I strongly recommend these two very good, recent books (freely available):

[1] Graph Representation Learning - William L. Hamilton https://www.cs.mcgill.ca/~wlh/grl_book/

[2] Deep Learning on Graphs - Yao Ma, Jilian Tang https://cse.msu.edu/~mayao4/dlg_book/

The new edition has been split in two parts. The pdf draft (921 pages) and python code [1] of the first part are now available. The table of contents of the second part is here [2].

From the preface:

"By Spring 2020, my draft of the second edition had swollen to about 1600 pages, and I was still not done. At this point, 3 major events happened. First, the COVID-19 pandemic struck, so I decided to “pivot” so I could spend most of my time on COVID-19 modeling. Second, MIT Press told me they could not publish a 1600 page book, and that I would need to split it into two volumes. Third, I decided to recruit several colleagues to help me finish the last ∼ 15% of “missing content”. (See acknowledgements below.)

The result is two new books, “Probabilistic Machine Learning: An Introduction”, which you are currently reading, and “Probabilistic Machine Learning: Advanced Topics”, which is the sequel to this book [Mur22].

Together these two books attempt to present a fairly broad coverage of the field of ML c. 2020, using the same unifying lens of probabilistic modeling and Bayesian decision theory that I used in the first book. Most of the content from the first book has been reused, but it is now split fairly evenly between the two new books. In addition, each book has lots of new material, covering some topics from deep learning, but also advances in other parts of the field, such as generative models, variational inference and reinforcement learning. To make the book more self-contained and useful for students, I have also added some more background content, on topics such as optimization and linear algebra, that was omitted from the first book due to lack of space.

Another major change is that nearly all of the software now uses Python instead of Matlab."

[1] https://github.com/probml/pyprobml

[2] https://probml.github.io/pml-book/book2.html

He has shared part of his toolkit, namely the exporting and syncing of the Bear notes, and “link-janitor” for the backlinks [1]. Although I wouldn’t recommend it in general - it’s a brittle prototype. And right now there are better tools out there (linked above) many of which appeared in the last few months.

[1] https://www.reddit.com/r/bearapp/comments/enbk65/sharing_a_s...

Right now, Andy is perhaps the most sophisticated thinker in this space sharing his insights and prototypes (meta-knowledge work, backlinked evergreen notes, spaced repetition, new UX/UI for these systems, etc). Here's some additional pointers:

- Andy livestreamed a demo of him on a typical work session: https://www.youtube.com/watch?v=DGcs4tyey18

- this Patreon post explains in greater lenght his OS-level spaced repetition approach: https://www.patreon.com/posts/bringing-ideas-36925173

- Andy is working on a prototype of that system, called Orbit, which might be available soon: https://twitter.com/withorbit

- in regards to his specific writing/thinking system, here's a couple more clarifications: https://notes.andymatuschak.org/z4AX7pHAu5uUfmrq4K4zig9x8jmm... https://notes.andymatuschak.org/z6f6xgGG4NKjkA5NA1kDd46whJh2...

- Obsidian has a plug-in which replicates the sliding panes of Andy's notes: https://forum.obsidian.md/t/andy-matuschak-mode-v2-7-updated...

I think the space of graph/backlinked personal notes/knowledge systems is taking off [1], with many solutions free and open-source. (Of that list, many have spaced-repetition plug-ins not referenced there.) It will be interesting to how the field matures in a couple of years.

[1] https://www.notion.so/db13644f08144495ad9877f217a161a1?v=ff6...

I like these two quotes of Knuth where he lets us know how hard he worked.

--

From this small interview [1]:

"When I'm working on a research problem I generally begin by filling dozens of sheets of scratch paper with partial calculations. When I eventually get to a point where I can think about the problem while swimming, then I'm often ready to solve it."

--

From this other interview [2]:

"So I went to Case, and the Dean of Case says to us, says, it’s a all men’s school, says, “Men, look at, look to the person on your left, and the person on your right. One of you isn’t going to be here next year; one of you is going to fail.” So I get to Case, and again I’m studying all the time, working really hard on my classes, and so for that I had to be kind of a machine.

I, the calculus book that I had, in high school we — in high school, as I said, our math program wasn’t much, and I had never heard of calculus until I got to college. But the calculus book that we had was great, and in the back of the book there were supplementary problems that weren’t, you know, that weren’t assigned by the teacher. The teacher would assign, so this was a famous calculus text by a man named George Thomas, and I mention it especially because it was one of the first books published by Addison-Wesley, and I loved this calculus book so much that later I chose Addison-Wesley to be the publisher of my own book.

But Thomas’s Calculus would have the text, then would have problems, and our teacher would assign, say, the even numbered problems, or something like that. I would also do the odd numbered problems. In the back of Thomas’s book he had supplementary problems, the teacher didn’t assign the supplementary problems; I worked the supplementary problems. I was, you know, I was scared I wouldn’t learn calculus, so I worked hard on it, and it turned out that of course it took me longer to solve all these problems than the kids who were only working on what was assigned, at first. But after a year, I could do all of those problems in the same time as my classmates were doing the assigned problems, and after that I could just coast in mathematics, because I’d learned how to solve problems. So it was good that I was scared, in a way that I, you know, that made me start strong, and then I could coast afterwards, rather than always climbing and being on a lower part of the learning curve."

[1] http://authenticinquirymaths.blogspot.pt/2015/11/maths-in-sc....

[2] Transcript from here: https://github.com/kragen/knuth-interview-2006

Here's Alain Connes, Fields medalist, on how a mathematician works and should read a book [0]:

"To understand any subject, above all, a mathematician SHOULD NOT pick up a book and read it.

It is the worst error!

No, a mathematician needs to look in a book, and to read it backwards. Then, he sees the statement of a theorem. And, well, he goes for a walk. And, above all, he does not look at the book.

He says, "How the hell could I prove this?"

He goes for his walk, he takes two hours ... He comes back and he has thought about how he would have proved it. He looks at the book. The proof is 10 pages long. 99% of the proof, pff, doesn't matter.

Tak!, here's the idea!

But this idea, on paper, it looks the same as everything else that is written. But there is a place, where this little thing is written, that will immediately translate in his brain through a complete change of mental image that will make the proof.

So, this is how we operate. Well, at least some of us. Math is not learned in a book, it cannot be read from a book. There is something active about it, tremendously active.

[...]

It's a personal, individual work."

[0] https://www.youtube.com/watch?v=9qlqVEUgdgo

Don't forget Gaius Diocles, the roman charioteer [1]:

"His winnings reportedly totaled 35,863,120 sesterces, allegedly, over $15 billion in today’s dollars, an amount which could provide a year's supply of grain to the entire city of Rome, or pay the Roman army at its height for a fifth of a year. Classics professor Peter Struck describes him as "the best paid athlete of all time"."

[1] https://en.m.wikipedia.org/wiki/Gaius_Appuleius_Diocles

If you are interested in learning about GAN, checkout GANLab, by Google Brain and others:

https://poloclub.github.io/ganlab/

It's similar to tensorflow playground, but for GAN:

https://playground.tensorflow.org/

I think John Bell discovered his revolutionary Bell's theorems of quantum mechanics in a similar way.

I can't find it now in a quick search, but I remember reading that he thought every physicist should devote something like 10% of their time thinking about the foundations of physics/quantum mechanics. (What would he do with 100% of his time?)

It's always going to take a few years.

The global fleet is about 2 billion passenger and commercial vehicles, and the global yearly production is about 100 million. So even if all new cars sold from now are electric, it will take 20 years.

But who knows what kind of autonomous vehicles and other innovations we'll have in 20 years. Buckle up :)

This is the most complete list of Alan Kay's talks I found: https://tinlizzie.org/IA/index.php/Talks_by_Alan_Kay

This is a bit of a strange article.

First, "in his recent book" refers to his 2011 book [1]. And Christensen has been prophesying this general bankruptcy "in the next decade" since that time. [2]

In any case it's interesting to think about the larger argument of the future of traditional higher education in general versus online education.

Bryan Caplan's thesis that (the state should cut funding for higher education because) higher education is mostly about signalling 3 things is a good tool. He argues that higher education signals a combination of intelligence, conscientiousness and conformity. The combination of the 3 is crucial for the model. [3]

Online education, and more generally self-education, fails on the conformity side. Companies do not want in general to risk such non-conformists, when they can hire from a stream of fresh graduates (smart, hard-working and relatively conformist).

Also, I think the socialization, friendships and networking that happen in the university are extremely valuable and not easily replaced by online education (where and with who can a smart, driven 18 year old hang out while studying and learning for 4 years on MOOCs and textbooks?)

And in addition, I hope, traditional universities are starting to improve their teaching methods (eg, flipped classroom, peer instruction) to multiply the pedagogical and motivational value they offer vs MOOCs.

For online education to replace traditional higher ed, it might require taking into account these factors. Could something like workspaces for freelancers or remote workers - but for studying - replace the traditional institution and the above benefits? Such that, for example, you would not be seen as an extreme non-conformist by not enrolling in a university?

Also, outside the US, tuition costs is often much lower. An online STEM degree, say a certified online masters in software engineering such as coursera or edx, could easily be more expensive than regular (or even the best) university.

[1] https://www.amazon.com/Innovative-University-Changing-Higher...

[2] https://www.economist.com/international/2012/12/22/learning-...

[3] https://www.amazon.com/Case-against-Education-System-Waste/d...

(To be clear, he argues that from the individual's perspective, university is still net positive, if you have what it takes to finish the degree and don't get too much in debt. It's the state that should cut funding since it's inflating credentials.)

Anders Ericsson has replied to this meta-analysis [1], which in turn got a reply from McNamara et al [2].

In Ericsson's opinion/definition [1], deliberate practice is "individualized practice with training tasks (selected by a supervising teacher) with a clear performance goal and immediate informative feedback was associated with marked improvement"; and he argues "In contrast, Macnamara, Moreau, and Hambrick’s (2016, this issue) main meta-analysis examines the use of the term deliberate practice to refer to a much broader and less defined concept including virtually any type of sport-specific activity, such as group activities, watching games on television, and even play and competitions. Summing up every hour of any type of practice during an individual’s career implies that the impact of all types of practice activity on performance is equal—an assumption that I show is inconsistent with the evidence."

McNamara et al reply saying that evidence only accounts for a relatively small fraction of expert performance [2]: "we found that deliberate practice accounted for a sizeable amount of variance in sports performance (18%), but it left a much larger amount unexplained. Ericsson’s (2016, this issue) evaluation of our research is undercut by contradictions, omissions, and errors." They conclude that "The available evidence indicates that deliberate practice, though undeniably important, does not largely account for individual differences in expertise. Building on Ericsson’s pioneering work, the task now is to develop theories of expertise that include multiple factors."

[1] Summing up hours of any type of practice versus identifying optimal practice activities: Commentary on Macnamara, Moreau, & Hambrick (2016) http://journals.sagepub.com/doi/abs/10.1177/1745691616635600

[2] How Important Is Deliberate Practice? Reply to Ericsson (2016) http://journals.sagepub.com/doi/abs/10.1177/1745691616635614

Pearl's words from the Introduction of "BAYESIANISM AND CAUSALITY, OR, WHY I AM ONLY A HALF-BAYESIAN":

"I turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in the language of probabilities, and (iii) If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases.

Thirty years later, I am still a devout Bayesian in the sense of (i), but I now doubt the wisdom of (ii) and I know that, in general, (iii) is false."