canjobear

What in his speech would not apply to language models?

As grim as it gets, with rare glimpses of light.

> Professor McGaugh has commented many times on the sociology of science and how the dominant paradigm tends to be hostile or indifferent to alternate views

This should also set off crank alarms.

What makes you think the government can’t compel you to provide a password?

Because there is a proof! If you are interested in the conjecture, then you would be interested in the proof. If you are not interested in the proof, then you weren't really interested in the conjecture. You were interested in the good style of articles about the conjecture, or the friends you made along the way, etc.

It is of course possible that either (1) the statement of the theorem in Lean is busted or (2) there is a bug in Lean. But both of these seem to me to be lower probability than that the proof is correct. It's not just an e-mail from a crackpot. For someone who is actually interested in the problem, the probability that the proof is correct is high enough to warrant effort to understand it, or at least to learn enough Lean to check the theorem statement.

This is the key bit.

> But, back to the Partition Principle. I took a brief look at the preprint released by OpenAI (not at the Lean code, since I know very little of the actual usage of Lean, and that code was enormous). It sucked. It is unclear, muddled, and has a strange structure.

Incredible lack of curiosity. The Lean artifact shows that there is a proof. Maybe the natural language writeup sucks (maybe it doesn't even correspond to the Lean proof!) but the proof is there and if he were really interested in the problem he would try to understand it. Rather, his revealed preference is that what he's really interested in is good style in academic papers.

Just ignore the emails. Academics are already very good at that.

In practice this already happens, unfortunately.

> Nor have they flailed wildly at LLMs as if complaining enough about it will make them go away.

If you look at the comments on the blog post, you'll find plenty of this.

> such a random corpus

I see what you did there

> Mao came quite close to abolishing Chinese characters entirely during the Cultural Revolution

This was indeed the Communists’ original plan, but the real story is more interesting. Before taking over the country the plan was to replace characters with a pinyin-like system Latinxua Sin Wenz[1] and a whole generation in the north was taught to read this. After 1949, Mao was talked out of Romanization by none other than Stalin, who told him the Chinese language deserves its own script. Pinyin was designed after that, officially as a teaching tool, but probably by people who secretly still believed in more radical writing reform.

[1] https://en.wikipedia.org/wiki/Latinxua_Sin_Wenz

A refutation of that view: https://www.complexsystemspodcast.com/episodes/credit-card-r...

What was his reputation before OpenAI?

The point of TFA is that there are regularities you can exploit in the actually existing weights of machine learning systems, not in some hypothetically maximally efficient weights. The maximally efficient weights would indeed have no structure, but that’s not what anyone is working with.

The weights aren’t compressed. So there are interpretable redundancies in practice.

It was accepted to NeurIPS.

I’ve encountered several people now who consult LLMs constantly for nearly every decision. Some proportion of people will absolutely cede their thinking to these things, and some already have.

> What about cooking food? We've become entirely dependent on cooking food.

In evolutionary time, we lost the ability to digest raw meat because we offloaded so much of process of the digestion to the cultural technology of cooking.

Do you like the looks of a future where that happens but for thinking?

It is a collaboration across many institutions including those from Europe, the US, and Japan

His arguments apply generally to a large family of interpretability methods. Skimming suffices to figure out that the linked article is in that family.

Compression is still prediction, the predictions just might not be good.

His videos on the people this article is about https://youtu.be/9cwcN-mm4xs https://youtu.be/XcDmr2_fvNo

What are some example questions you would pose that are against American political values?

It was standard procedure to do invasive surgeries on newborns without anesthesia until the 1980s, on the theory that they can’t feel pain. https://hms.harvard.edu/news/long-life-early-pain

Midwives don’t participate in C-sections

I was clicking around in response to this article and found this video that explains the inspection paradox nicely. https://youtu.be/Jd1wNizPjoE

You can evaluate the logic for the decipherment step by step and make sure all the claims are justified. But the best test is to try the proposed decipherment against some new text and see if it makes sense. In the case of Linear A and the other remaining undeciphered scripts, there's not a lot of held-out text to test against, so it's tricky.

> Tom's work is credible enough that it's being reviewed by linguistics experts at Rutgers and Cambridge

What does this mean? Like he e-mailed it to some people at Rutgers and Cambridge? Or it's under some kind of non-anonymous peer review?