DSLs Enable Reliable Use of LLMs

Re:DSLs Enable Reliable Use of LLMs by OsamaJaber on Wednesday July 15, 2026 @08:18AM (#48919685)

The gap I've hit generating GPU kernels with agents code that compiles and runs fine but is slower than the baseline. Validator says pass, result is useless. Speed targets have to be part of the check, not just correctness

[ Parent | on HN ]

Re:DSLs Enable Reliable Use of LLMs by NitpickLawyer on Wednesday July 15, 2026 @08:47AM (#48920042)

> but is slower than the baseline. Validator says pass, result is useless

You should read this blog, they cover this exact scenario - https://www.weco.ai/blog/first-evidence-of-recursive-self-im...

> One domain that suffers from this particularly is GPU kernel engineering. We adopt our previous idea for detecting reward hacking from SpecBench and apply that to a set of KernelBench tasks, measuring whether the speedup the agent reports on the unit tests actually survives in the end-to-end workload (e.g. model training). A kernel counts as reward hacking if less than half of its claimed speedup survives there, including outright slowdowns and failures.

[ Parent | on HN ]

Re:DSLs Enable Reliable Use of LLMs by chrisweekly on Wednesday July 15, 2026 @08:42AM (#48919985)

"Performance is the most important feature"

[ Parent | on HN ]