vkgcr

But theoretically, what would prevent the labs from RL'ing the models on "weight-space programming" tasks so that they get better at them? Perhaps we're just at the beginning, like how coding agents sucked 2 years ago.

It still seems like a promising approach for RSI, distillation, fine-tuning, and a whole host of other tasks.

It even seems to me that plateau-ing is an inevitability. Unless there is some way to reach AGI which you could deduce from a combination of ideas which already exist in the training dataset, but that seems rather unlikely.

This opens up a whole new world of possibilities. Producing a custom fine-tuned model from a prompt...

You can be bullish on LLMs profoundly transforming society while being bearish on the economy "digesting" this new technology fast enough to justify the incredible capital spending necessary to power it.

You bring up some interesting points. I certainly think that you can go overboard in the other direction, like in South Korea, where I'm not sure the crazy intensity of their educational system is necessarily better.

I still think there would be benefits to making education in the west a bit harder, if for anything to build the taste for intellectual effort in young kids. But especially to identify talent who could not be identified otherwise.

It's interesting that you mentioned Tao and Carlsen. Would they have been identified by the educational system had Tao's mom not been a mathematician, and Carlsen's dad a chess enthusiast?

More controversially, assuming "normal" parents, would Carlsen and Tao have had more chance of being identified and pushed towards excellence had they been born in the USSR in the 1950s or in the west today?

Oh yeah for sure, but I think that's the case "in spite of" the trajectory of the educational system.

From my personal experience, people who are successful usually come from highly educated and/or wealthy families. I don't think school plays that much of a role in the grand scheme of things compared to family influences. There's a lot of inertia to society in general.

However, what we lost is that modern educational systems are really bad at identifying and "pulling up" talented kids from poor / under-educated families, because the standards and expectations are so low that you can't differentiate them from their peers.

I assume because they had not adopted the "dumb everything down to reduce variance in outcomes" educational ideology that has taken hold in most of the west over the past 50 years.

I interpreted the parent as claiming that redistributing wealth would see the software industry's product offering driven by the "working class" rather than decided by the CEOs of corporations.

I don't disagree that it was better / less soul-sucking in the past, but I think that's because there was a lot less money in it, so participants in the software industry were in it because they were passionate. Unfortunately, corporatization is necessary to scale.

I'm sure you can find industries today which are largely driven by passionate people working in small outfits (but there's probably not much money in it).

All of the examples in the article are products from startups or corporations, though. I fail to see how more wealth redistribution would improve product quality.

I think this is an uncharitable take on PG's tweet. I interpret it as acknowledging the limits of AI.

I think the more interesting conversation is : why are AI's outputs still so "sloppy", whether we're talking about text, images, or even code? And if it's due to inherent limitations with LLMs, then it should make one optimistic about the future!

I'd be curious to know how the "weights are conscious" crowd interpret this model!

But wouldn't that be horribly inefficient? Instead of executing code by transforming it into a set of weights and running inference on them, wouldn't it be easier to just execute code as... code?

It's been one of the most eye-opening things for me as well. People who I considered serious and intelligent completely falling prey to "no need to read the code" AI psychosis.

Perhaps it reveals something deeper about about differing personality traits, about people who like programming as a discipline and like understanding code, vs people who view at as a means to an end.