This is the same stupid argument trotted out repeatedly in the thread, and it doesn't hold up to reality. Would you care to explain how adding specifications helps a model that demonstrably ignores specifications? A model that cannot even handle the trivial specification of "the filename must be lowercase" is not going to get better when you add more specifications it ignores. I would actually guess that you're underspecifying relative to me. Perhaps you're happy with whatever slop you get and don't notice Luna's overwhelming incapability to actually provide what one requires, because you don't have requirements.

Wrong. What I want Luna to be able to do is ~deterministically make small edits or generations of 1~10 LoC according to a specification, and it literally cannot even do that. It violates specifications no matter how small you aim.

> there is nothing deterministic about using llm's for codegen

And if you're okay with that, you're more like a farmer eating the slop meant for his pigs than an engineer. Using Sol Max, you can get near-deterministic output for a much wider range of tasks and specifications. Not all specifications; at a certain degree of sufficiently detailed specification it will begin failing again. But some, which is better than none.

you just aren't using it right man. stick to the big guns, you apparently need it.

> you're expecting to use it without steering

If I have to manually "steer" every mistake it makes on every trivial task, instead of being able to deterministically eliminate classes of failures so that fixing the task one time ensures that failure never happens again on similar future tasks, it is in fact faster to just type the fucking code myself.

but your claim was about file name invariants when generating code, especially small amounts of code, which is not what they were doing with DeepSWE, so you've confused two different things.

anyways, this is a pointless argument. if you can't figure out how to effectively use smaller models in your workflow, fine. but don't come on here talking shit about people who can. have fun slopping out yet another ai harness at your startup.

shhhhhhh, not so loud!