DSPy?

I work at .txt, where we work on exactly this. We use structured generation to make programmable language model systems. Here's two examples I worte + love.

One makes a (sort of) internally consistent infinite world generator with RAG + Outlines:

https://github.com/dottxt-ai/demos/tree/main/lore-generator

This one makes two characters up and then builds a simple combat loop where they fight each other:

https://github.com/dottxt-ai/demos/tree/main/nouscon-24

Both are simple examples of basic programming with language model outputs.