qbit42

make no mistakes

mostly jk, although agents themselves are of course getting better at detecting whether a formal spec is faithful to a natural language spec

For what it's worth, most mathematicians are paid to teach, and the amount of research funding going into mathematics is relatively small. Plus, there are rare but important broader advancements due to pure and applied math that are hard to predict in advance (e.g., cryptography, compressed sensing).

Mathematicians also decide what to prove in their research - taste is not a skill we have RLVR'd away (yet, not claiming this is fundamental).

Of course, the field is being fundamentally transformed, it will probably end up looking more like experimental science in the future.

In his recent interviews, he says he has very little time to do math these days outside of research meetings with his PhD students and postdocs.

Is the use of "generalization" like this standard in mech interpretability? Seems undesirable given the typical definition in ML.

It just looked to me like the GP hadn't read the article and assumed it was a post by Tao. You are right about Tao of course, no objections there.

This post isn't by Tao.

The quant firms are heavy AI investors I believe.

You can also brute force the existence of a proof...

It is true that the biggest splashes have been from counterexamples and such, but I have seen many smaller examples of proving positive results in my own field.

Perhaps not superhuman yet but sufficient to completely upend the status quo. And I'm supposed to believe that this won't change in a year? or two?

Maybe, but what remains of the human professional mathematics will be unrecognizable (at least for those without tenure I guess). All of our credit assignment systems are breaking and access to computational/financial resources is becoming way more important. Math has been one of the most open academic fields but everyone is becoming afraid of sharing their ideas. Personally, my job has slowly been moving from open-ended brainstorming to prompting/digesting LLM output (or LLM output transmitted by grad students). And the students are so demoralized! Also, the academic funding structures which have supported math departments are looking less and less stable - I imagine many of them will shrink.

And getting paid…

I heard Hodge conjecture? Third-hand rumor though...

It's not impossible, but I also know of no instances of an AI being told to prove something in Lean and exploiting such tricks. Surely Anthropic also had some agents looking for issues with the generated proofs. Personally, I am also comfortable trusting Kevin Buzzard, who was leading the human team aiming to formalize FLT and discussed this a bit on his blog.

Finally, the good thing about Lean is that if anyone ever finds a new compiler bug (which, by the way, are being searched for extensively using AI), you can correct the bug and recompile any old proofs of which you are suspicious. Any tricks in a false proof must be exploiting a bug in the Lean compiler, so as we increase trust in the compiler over time we also increase trust in every previously compiled proof.

I agree it's not a 100% guarantee, but in this case the human proof is well-understood and written about by many experts, so I think Claude had plenty of material to work with. Even if the task was enormous, I don't think any of the individual steps are out of the scope of what we have seen from current AI tools.

I have never seen an AI or a human produce a false proof without explicitly using weird meta programming tricks that are very suspicious. No "good faith" Lean proofs have every been shown faulty, to the best of my knowledge. While the risk is non-zero, many of the AI companies are also trying to find bugs in the Lean kernel, so it is becoming very well stress-tested.

You just have to trust the statement and the lean compiler, not the proof. The compiler certainly still has remaining bugs, but I have never seen a bug leading to a false proof in good faith, only via obscure meta programming tricks. The nice thing is that the multiple versions of the compiler are constantly being stress tested. Still, there is plenty of work that could be done to make the compiler more trustworthy / easier to verify.

Watermarking just alters the pseudorandom number generator. If "I like turtles" was previously the response to your prompt with probability 100%, it will still be so. This is why watermarking is only effective for long strings of text

I think the old reddit redirect extension gets around this somehow, that has worked for me lately (occasionally with some toggling of the extension). Although maybe it's best to just use less reddit tbh

Are legumes not "real food"?

I somewhat agree, and I have not stopped eating beef entirely. I think that forbidding is overkill but agree that lots of related subsidies should be eliminated (honestly, if meat were just priced w/o current subsidies, we'd be in far better shape).

But you have to have some public consensus or the government will never implement such changes due to their unpopularity. This is why I think promoting some reduction in individual meat consumption is still valuable - it is good for average people to reflect on their consumption or they will vote out anyone who attempts to change it.

I don't think that the focus should be on individual blame (even just for the reason that this is politically nonviable). But you at least need to amass some collective awareness.

I mean, it's a collective action problem. Individual consumption of normal people is a huge contributor to global energy demands / pollution / etc. You don't have to change your behavior but if no one does in the coming decades then we're pretty fucked.

Not to let the 1% off the hook. Por que no los dos

There is not a single foundation - you can choose. The differences are rarely important for working mathematicians though. Most know enough of ZFC to get by and ignore foundations tbh

People talk on the phone all the time on every US metro I’ve tried. Seems fine if they’re not on speaker or raising their voice

It is marketing, they are a shady company, and yet, if someone had access to these results before today's modern AI tools, they could get tenure at any university in the world.

So this is a bit of a hot take for HN, but while my life is dramatically more convenient due to smartphones, “better” is actually not obvious to me. Some things are better, some are worse.

Anyway, I don’t think anything I said was even framed morally (although of course I have my personal opinions). My prediction is just that the amount and average quality of paid creative work is going to massively decline. To me, the scale difference matters a lot (compared to impact of previous tech). I hope I’m wrong.

I agree with some of your characterizations here, but I don’t think it is fair to say that if you are not currently a professional artist, you were never a true artist. People get unlucky, have families to support, etc

Sure, this mirrors history, technology is always transforming economics.

I guess my point is that there is a decent sized middle class of creative professionals right now, and I expect that the number of them will go down significantly due to AI. Seems like art will become more and more restricted to hobbyists and the lucky few who achieve mainstream success.

This is kind of what happened with photography. But I think the scale is much bigger, affecting all creative industries.

It was already bad, but I think there is still room for it to get a lot worse unfortunately.

They are not always the same but there is more overlap than you would think. One place that I know is being hit hard is Nashville (no comment on what this implies about the state of US country music). There are a lot of local musicians with their own (small) bands whose primary income came from (relatively low quality, compared to their own music) song-writing, playing back-up instrumentals/vocals for studios, stuff like that. This income stream is shrinking rapidly.

AI is destroying the economics which allowed for a sizable middle class of artists. The issue is that many are paid for their art mostly for its aesthetic rather than artistic value. This isn’t the most creatively fulfilling, but it previously allowed many artists to make a living while refining their skills, often enabling them to pursue their real creative ambitions on the side.

I think shade is a big factor here. Trees can really cool down a neighborhood.

I found replacements for the Chrome extensions I was using and switched to Zen, which builds off of Firefox and closely resembles Arc (RIP).

It might not be the best security idea to rely on a relatively obscure browser like this, but I find it very pleasant to use.