Executing programs inside transformers with exponentially faster inference

Re:Executing programs inside transformers with exponentially... by deviation on Friday March 13, 2026 @05:36AM (#47362293)

I really liked the article, but food for thought: is a transformer that offloads computation to python really that different from Python code being read and then executed by a compiler?

Both examples are of a system we created to abstract most of the hard work.

I think a more important concept here is that the term "AI" has a lot of built-in assumptions, one of which being that it is (or will be) super intelligent, and so folks like the author here think (correctly) that it's important for the AI to be actually doing the work itself.

[ Parent | on HN ]

Re:Executing programs inside transformers with exponentially... by krackers on Saturday March 14, 2026 @03:06AM (#47374093)

There's one difference that if a program is run as tool call, the internal states and control flow are not visible to the LLM. You can imagine this being useful for "debugging" in a meta-sense, the same way humans can use debuggers to figure out where something went awry it might be useful for the LLM to "simulate" something and have access to the execution trace.

Of course you can also just simulate this by peppering your code with print statements, so maybe it's not that useful in the end after all.

[ Parent | on HN ]