jhrmnn

I've personally experienced three big jumps in LLM capability: o3, opus 4.6, and opus 5.5. I could go back to opus 4.6, but it did require way more involvement from my end.

Are there simply two cultures among programmers? Along similar lines as the two cultures of mathematics? One motivated by the process itself, by esthetics, by abstractions, ideas. The other by solving problems. And then obviously these two cultures react very differently to AI.

It all hinges on the definition of “progress”. The debate of the past month is all about questioning whether formally proving outstanding unproven theorems without human understanding constitutes progress. This is quite different from solving diseases.

Cancer is the price we pay for complex multicellular life. It’s not a failsafe, it’s the failure of all failsafes to regulate cellular growth within an organism.

No clue what’s the consensus on this but my internal mental model is absolutely that LLM AI is pure fast mode, no slow mode. The “reasoning” loops are an attempt to mimic the slow mode but ultimately it doesn’t really work. I’m curious about the recent maths advances though, they seem to possibly challenge this.

Take the tax system and the state out of the picture. How does a mathematician convince his fellow citizens that they should fund their work? Newton didn’t have to do that, he had income from other sources and did most of his mathematical research as a hobby

It seems only natural for nature to have it like this. It’d be hard for evolution to optimize things if the genotype to phenotype mapping was sharp, with phase transitions and such

It’s written from a perspective of decades, not years

It’s actually a fairly precise use of the word, just not in the very specific and narrow sense in which it has been used for AI.

Working with AI on science (not LLMs though), couldn't agree more.

I think it’s more likely that that’s because no one tried to solve such problems with them before (OpenAI apparently started working in Navier-Stokes after a rumour that someone seriously advanced the problem with AI) plus improvements in orchestration. Fair, the latter could be as dangerous as stronger models.

I can almost see two branches of mathematics developing. One which is human-understandable, the other formally verified. I assume the latter is a strict superset of the former?

The halting problem on steroids?

This starts to feel like chess engines. It’s obvious their play is superior but it’s impossible for humans to understand the moves.

Reliability is one property of a product. And it’s quantitative, not qualitative, so it all comes down to trade-offs

The Dutch auction vs lottery are two ways to solve scalping. They differ in what one considers fair.

When no work is safe from mechanization, surely the value of labor wrt capital must fall, and the societal pressure on redistribution will rise. The ultimate outcome of technological progress is either extreme inequality or massive redistribution

This can’t be sustainable, there must be a limit in human biology as to how complex jobs we can handle. More and more people will fall under that threshold.

Is this a different route to the universal basic income scenario?

Nothing, it’s that same story again. Industrialization turned peasants to blue collar workers by mechanizing agriculture. Then blue collar workers were turned to white collar workers by mechanizing all manual labor. Now AI is coming for white collar workers by mechanizing intellectual labor. The big question is what will white collar workers turn into.

So most training data would be grey and a little bit coloured? Ok, that sounds plausible. But then maybe they tried and the current models get it already right 99.99% of the time, so observing any improvement is very hard.

Because then the training data would have to be coloured

Interesting. The debate about whether the artist matters in perceiving a piece of art is very old. You don’t seem to consider the possibility that the artist’s intent matters when listening to music. For me it absolutely does. As the AI has no intent (agency), the AI music is void of any value to me.

Forget the government, what about your fellow humans? Is defending your country an obligation towards your government or towards your neighbor?

It’s interesting to read the discussion here through the lens of obligations vs rights. It would seem the rights are definitely winning.

The cooperative and competitive sides of our soul fighting it out in a single situation

I find it sad that while technological progress is seen almost as a given by virtually everyone, moral progress is often not even an aspiration

Think of a woodworking project. Compare doing everything old-school by hand vs using modern tools to go faster. Think about the end product being just an item with a function vs it having some design value or even craftsmanship value. Does the parallel work?

But individual behavior is not about preventing climate change, it’s about doing what’s right. It’s wrong to pollute the environment, one way or another. A single person not stealing won’t reduce the crime rate in a country yet it is the right thing to do.

This depends very much on what "practical purposes" are. For almost all conceivable technology, relativistic quantum mechanics for electrons and light, ie QED, is sufficient fundamental theory. This is unlike before quantum mechanics, when we basically didn't have fundamental laws for chemistry and solid-state physics.