News for hackers. Stuff that matters.
Re:" and the proof is checked in Lean" by wodenokoto on Monday November 16, 2026 @05:56AM (#50516209)
Isn’t “proof is checked in Lean” kinda similar to assuming your code works just because it compiles.

Sure, the code works, but does it work at doing what you want?

Sure, the proof is valid, but does it prove what you set out to prove?

Re:" and the proof is checked in Lean" by nifets on Monday November 16, 2026 @06:04AM (#50516250)
it's not really similar at all.

proof checked in lean => proof of statement requires only that the statement is specified correctly (generally very easy to do) and that no lean kernel bugs were abused

code compiles => code satisfies requirements requires checking that all (inputs, outputs) of your program match the expected behaviour which is wildly intractable to verify in general

Re:" and the proof is checked in Lean" by oulipo on Monday November 16, 2026 @06:05AM (#50516253)
If the proof is valid, it DOES prove what your theorem is saying (except for rare verification kernel bugs). The issue is mostly: did you correctly formalize and check the specification of your theorem itself...
Re:" and the proof is checked in Lean" by j16sdiz on Monday November 16, 2026 @06:23AM (#50516354)
it is more like "unit test pass"

Fresh today, I asked llm to fix a unit test for me, it replaced the test with no-op.

Re:" and the proof is checked in Lean" by bananaflag on Monday November 16, 2026 @06:36AM (#50516441)
No, because proofs are programs and theorems are types. It is enough to construct a function of the needed type to have a proof of the needed theorem. You don't need to look through all the code to see that the function has the desired type.
Re:" and the proof is checked in Lean" by lordnacho on Monday November 16, 2026 @07:34AM (#50516935)
This sounds interesting enough that I think a link would be useful
Re:" and the proof is checked in Lean" by RossBencina on Monday November 16, 2026 @07:42AM (#50517002)
Re:" and the proof is checked in Lean" by Sharlin on Monday November 16, 2026 @07:53AM (#50517089)
You have to be sure that the formalized theorem matches the informally stated theorem. Which is what GP meant and what the article is about. A program does what is written, not what is meant; a Lean program proves what is written, not what is meant.
Re:" and the proof is checked in Lean" by bananaflag on Monday November 16, 2026 @08:17AM (#50517313)
I agree with that. I never said you don't have to check the statement.
Re:" and the proof is checked in Lean" by fweimer on Monday November 16, 2026 @11:05AM (#50519129)
That's like saying that all memory-safety programs are written in Rust.

There is no fundamental reason why mathematics should be formalized using concepts that are alien to most mathematicians, in the way that type theory is.

(I guess Isabelle/HOL is the Java of formalized mathematics, in the sense that for reasons that are difficult to understand, memory safety of Java-implemented technology does not count.)

Re:" and the proof is checked in Lean" by bananaflag on Monday November 16, 2026 @12:55PM (#50520573)
Re:" and the proof is checked in Lean" by mentalgear on Monday November 16, 2026 @12:33PM (#50520254)
That's basically what Typescript also gives you, and it isn't bug free either.

You can proof a logic, but it does not proof that it is the logic that would keep the program secure and bug-free, only that it is internally consistent. So it helps, but by no means a silver bullet !

Re:" and the proof is checked in Lean" by fancyfredbot on Monday November 16, 2026 @06:57AM (#50516606)
I think it's a reasonable point but a bad analogy.

It's more like assuming that because your code is formally proven correct the client will be happy with it.

The code can be right but it may still do the wrong thing.

Re:" and the proof is checked in Lean" by pasquinelli on Monday November 16, 2026 @07:24AM (#50516855)
you just said the same thing
Re:" and the proof is checked in Lean" by fancyfredbot on Monday November 16, 2026 @11:47AM (#50519663)
OP said it's like saying the code compiles, but I said formally proven correct.

Formally proven correct is a much stronger guarantee that the code meets its spec than a successful compilation.

Re:" and the proof is checked in Lean" by pasquinelli on Monday November 16, 2026 @01:07PM (#50520737)
and op's question was "but how do you know the spec is what you want." making the guarantee stronger doesn't fix the problem. you can't fly by pulling your feet up no matter how strong you are.
Re:" and the proof is checked in Lean" by dgellow on Monday November 16, 2026 @07:16AM (#50516778)
I mean, in some strange way, yes. Because your type system is basically a set of claims that are proven by your compiler. But the analogy doesn’t go further than that. The running program itself is the proof
Re:" and the proof is checked in Lean" by throwawayffffas on Monday November 16, 2026 @07:44AM (#50517021)
The gist of the article is that it's worse.

That it's the equivalent of saying it's full moon tonight because I wrote this program.

And he states that you have to check that:

1. The statement the author is making corresponds to the statement in the lean program.

2. The lean program compiles.

3. There is no shenanigans hidden in the lean program to disguise a trivial proof as the one stated.

4. The program does not exploit bugs in lean.

Re:" and the proof is checked in Lean" by csande17 on Monday November 16, 2026 @08:12AM (#50517263)
[delayed]
Re:" and the proof is checked in Lean" by lumost on Monday November 16, 2026 @11:10AM (#50519196)
Saying that a "formally verified" program rules out any bugs is a bit of a reach. Formal verification only checks that the properties you claim are true are actually true in the absence of bugs in the proof logic.

Example bug

A verified network program ensures that "data from two addresses are never commingled", but it does not verify that the NIC can't overwrite control flow memory opening up RCE attacks/other memory reads.

It certainly is harder to make bugs in formally verified software than the alternative, but it is not bug free.

Re:" and the proof is checked in Lean" by erichocean on Monday November 16, 2026 @12:49PM (#50520487)
> but it does not verify that the NIC can't overwrite control flow memory

Yes, Lean verifies the algorithm being verified, not hardware or someone else's software.

Re:" and the proof is checked in Lean" by paulddraper on Monday November 16, 2026 @10:24AM (#50518569)
That’s pretty much the same as compiling a program.

1. Yes.

2. Yes.

3. This is “you took 10 KLOC to do what 1 could have.”

4. Yes.

Re:" and the proof is checked in Lean" by dozerly on Monday November 16, 2026 @12:52PM (#50520514)
> the program does not exploit bugs in lean

Being fairly naive to Lean in general, this always seemed to me like a clear flaw of using Lean solutions as proof of hypotheses

Re:" and the proof is checked in Lean" by dwohnitmok on Monday November 16, 2026 @01:01PM (#50520651)
> 3. There is no shenanigans hidden in the lean program to disguise a trivial proof as the one stated.

To be clear, this is basically a case of bad defaults in Lean. As Buzzard says, the fix is literally one command.

> The one-line answer: `lake comparator`.

Re:" and the proof is checked in Lean" by program_whiz on Monday November 16, 2026 @08:15AM (#50517288)
Reading the friendly article:

It is worth noting that of the 290 OpenAI theorems which come with Lean code, around 38 of them have the property that the Lean code does not even state, let alone prove, the main theorem of the paper.

So even if LEAN is 100% bug-free, its still on the programmer to correctly express the logic and proofs, or they could be proving completely unrelated facts.

Re:" and the proof is checked in Lean" by throwawayffffas on Monday November 16, 2026 @09:47AM (#50518159)
They could be proving completely unrelated stuff. Or even entirely trivial things like 1 + 1 = 2 over multiple millions of lines of lean code.
Re:" and the proof is checked in Lean" by YeGoblynQueenne on Monday November 16, 2026 @11:58AM (#50519805)
[delayed]
Re:" and the proof is checked in Lean" by AnimalMuppet on Monday November 16, 2026 @12:50PM (#50520492)
Isn't that pretty much by definition? 2 is defined as the number that is 1 more than 1.
Re:" and the proof is checked in Lean" by bunderbunder on Monday November 16, 2026 @09:55AM (#50518233)
It also speaks to why I remain skeptical of relying too much on coding agents for code review. They are still pretty iffy on implementing the right thing, and even worse at detecting when a PR didn’t implement the right thing. Ignoring that reality is how you do like Anthropic did and increase your rate of merging changes from 2500/day to 30,000/day without a corresponding increase in the rate at which you deliver useful work product.
Re:" and the proof is checked in Lean" by skybrian on Monday November 16, 2026 @10:44AM (#50518828)
OpenAI did a data dump of work in progress, leaving it to the mathematics community to make sense of it. I don’t think they claimed that there were Lean proofs for everything? Did they warn that some Lean proofs were incomplete?
Re:" and the proof is checked in Lean" by suddenlybananas on Monday November 16, 2026 @10:53AM (#50518951)
Why did they stop in the middle anyway? Was there a justification given for that?
Re:" and the proof is checked in Lean" by bunderbunder on Monday November 16, 2026 @10:59AM (#50519044)
The cynical answer is that their primary goal was never to meaningfully contribute to mathematics. It was to pick some low-hanging fruit for the marketing department. So they did the 10% of the work that generates 90% of the buzz.
Re:" and the proof is checked in Lean" by skybrian on Monday November 16, 2026 @11:15AM (#50519257)
It’s more likely that their goal was internal research, to test a new LLM before releasing it. But, then they had something that mathematicians would be interested in, so they threw a data dump over the wall. It’s a rather rude way to do it, but this isn’t a company that particularly cares about the things that mathematicians care about.

OpenAI no longer needs to do anything to get publicity. There are some organizations that are so famous that everything they do is newsworthy and anything they do will be covered whether they like it or not. Remaining low-profile is no longer an option.

The people claiming that everything is marketing haven’t come to grips with the inequality of fame.

Re:" and the proof is checked in Lean" by yk on Monday November 16, 2026 @08:26AM (#50517382)
On a technical level, yes. Though lean is build such that "it compiles" is a very strong statement about mathematics while most programming languages are not build to provide a very strong statement about correctness of the program. It is perhaps more similar to checking an equation numerically.
Re:" and the proof is checked in Lean" by Gabriel54 on Monday November 16, 2026 @09:15AM (#50517814)
For relatively "simple" statements in mathematics it is pretty easy to learn enough Lean to verify that the Lean statement correctly encodes the intention of the mathematics. It gets more difficult in certain areas of mathematics where there are many layers of definitions and they haven't been standardized in Lean yet.
Re:" and the proof is checked in Lean" by paulddraper on Monday November 16, 2026 @10:20AM (#50518527)
Yes it’s a good analogy.

You have to watch out for

1. Translation. Do the author’s claims about the program/theorem match the actual code?

2. Do you employ “shenanigans” to circumvent the checker? Bugs, unsafe operations. — Lean is a very strict checker as compilers go. Some compilers intentionally say “buyer beware.” It’s annoying to check for integer overflow in every operation. Lean doesn’t. But that still doesn’t mean it has zero bugs, in the framework or axioms.