Davidzheng

To get to the other side, obviously.

I think anthropic one is edited by humans, OpenAI drop just looks like how Astra writes (i know it's probably a different model producing this but still)

Maybe it's arrogant to say as someone working in pure math but I highly doubt this is the cause

Makes me sad that the homotopical foundation people never got their shit together and now formalization is so mature but it's still done in set theoretic foundations.

they were never meant to stay as manufacturers on the sidelines

I mean, the models (starting with Bel clearly and Astra probably--though Astra probably is worse than top humans in some areas) are superhuman at math.

I don't understand. The AI-math guidance counsel has recently explicitly argued against what you say in paragraph two, and which is why for the Kakeya problem there was no group formed. The different communities have given different suggestions. I'm not sure how much I agree with the first paragraph either given these facts.

I think it should be allowed yes. I don't think our strategy is hoping the agents never learn that misalignment is an option.

The answer, as Noam Brown explained in the Dwarkesh interview, is that it is not emergent at all. They carefully selected reward functions in RL to incentivize cooperation in the last year in order to develop multi-agent systems which are faster and more reliable. However, I guess after it is baked into the model to cooperate, they can sometimes override instructions not to.

I highly doubt this will happen.

depends on what you think shrink means. If you mean decreasing, that's false even eventually.

I have to say, if you looked at the graphs of the last reports, 1.5C was never going to happen.

yeah agreed.

But then you can start with a solution and try to make an understandable one.

Happy to hear that yes! I hope more people do learn about math from this !

If you had halfway to one of these papers you would be anyhow be in the top echelons of math phds so probably you have less to worry than most!

So they can try to advance it even more

I'm not sure a superintelligence needs "funds" to take over the world.

I'm sorry, but what?

If you appreciate beauty and don't care about competing then these releases are purely good. Because you are not competing, so you aren't hurt by speed. And you are appreciating so you can appreciate more stuff.

sorry if I am misunderstanding (probably I am)

I think now it's a good time for people to work on the Riemann hypothesis. There are actually a lot of ideas, no one knows if any of them goes anywhere--but I don't think it's completely impossible to make progress especially with new tools haha.

You're not alone, even among mathematicians.

I do think something does happen/change when we reach superintelligence.

Maybe 4 year olds already have the brain power to learn to drive if they so desired.

I think I'm mostly on your side (also I believed early it would get here), but what does it mean to "solve the field"? I'm pretty much as bullish as you get on AI but I'm sure I can come up with questions AI cannot solve. I think the space of problems in math is so large and the distribution of proof lengths so heavy-tailed, there's probably no way without dyson-sphereing the sun to solve all easy to state problems.

surely the LLM can do that. It is RL'd against some reward, if the known strategies are clearly suboptimal with easy improvement, it'll find it most likely

It's honestly just good at math. Even at theory building i wouldn't put it below 90 percentile. Just on some things it's superhuman already and some not yet.

Absolutely not the right way to deal with a rogue super intelligence. At a minimum it could implement some dead man switch when it's out and knows about impending kill switch

I believe this is false. They hack bc hacking has nontrivial initial probability (within range of behavior seen in pretraining) and that probability is being heavily rewarded in RL post training

Yeah I agree, probably can't survive off regular consumer hardware or most non AI datacenters.

Market equilibrium probably tends to give 100% resource to ai building bc it leads to most returns and all human labor goes to zero.

Caveat I know nothing about economics lol--but this is my uninformed opinion.