Re:.md File Is Not a Specification: Using Formal Analysis to...
by jayaprabhakar on Thursday October 08, 2026 @05:23PM
(#50012506)
The difference is accuracy, cost, time and more importantly consistency.
In case you noticed, a year ago, LLMs could not reliably count the number of 'R's in strawberry. Now they all do well. Guess how? Instead of training LLMs to do this, it was easier for them to write a small python script and run that. That solved the problem once and for all.
The same thing here, instead of just using LLMs and keep grilling repeatedly, there is a higher chance of getting a workable solution quickly. The best part about formal methods here is, it is self validating.
It checks in seconds, what would have taken hours or even days with LLM only flow.
Re:.md File Is Not a Specification: Using Formal Analysis to...
by jayaprabhakar on Thursday October 08, 2026 @06:44PM
(#50013488)
The overlapping part of what FizzBee and Kiro is we formalize the requirements first and do formal analysis on them to identify various requirements issues.
However the technique is significantly different. Kiro's post says they use predicate logic. Whereas FizzBee uses Dynamic Logic. So, Kiro's approach cannot find many issues. Let us take the same example from the fizzbee blog. FizzBee found the issue as linked in the blog:
But Kiro's approach would say, it is both consistent and complete. That is, R2 + R2b => R3 in this case.
-----
Another thing is testability. FizzBee's approach checks for testability without LLM deterministically. And it naturally produces extensive test cases, but with Kiro it doesn't. It needs more LLM use to convert them to test cases.
Re:.md File Is Not a Specification: Using Formal Analysis to...
by pbronez on Thursday October 08, 2026 @06:28PM
(#50013313)
The related https://fizzbee.ai/ tool is pretty neat. It's similar to /grillme but with additional formalism. Not sure if the resulting specs are definitively better, if only because the FizzBee code is harder for me to decipher.
They typically capture the requirements, design, and implementation plan in Markdown files. But is that Markdown file actually a specification?
This article explores how formal analysis can uncover requirements gaps that are easy to miss.
I'm not sure if this does much more than a grillme skill and then poking an agent to do the work.
In case you noticed, a year ago, LLMs could not reliably count the number of 'R's in strawberry. Now they all do well. Guess how? Instead of training LLMs to do this, it was easier for them to write a small python script and run that. That solved the problem once and for all.
The same thing here, instead of just using LLMs and keep grilling repeatedly, there is a higher chance of getting a workable solution quickly. The best part about formal methods here is, it is self validating. It checks in seconds, what would have taken hours or even days with LLM only flow.
Bpmn, tla+, event-b, P and ModP are all likely contenders to jump in popularity and mainstream swe worlds
Correctness-by-Construction, Models and Proofs go together!
However the technique is significantly different. Kiro's post says they use predicate logic. Whereas FizzBee uses Dynamic Logic. So, Kiro's approach cannot find many issues. Let us take the same example from the fizzbee blog. FizzBee found the issue as linked in the blog:
https://blog.fizzbee.ai/formal-analysis-in-requirements-spec...
But Kiro's approach would say, it is both consistent and complete. That is, R2 + R2b => R3 in this case.
-----
Another thing is testability. FizzBee's approach checks for testability without LLM deterministically. And it naturally produces extensive test cases, but with Kiro it doesn't. It needs more LLM use to convert them to test cases.