If DSPy is so great, why isn't anyone using it?

Re:If DSPy is so great, why isn't anyone using it? by memothon on Monday March 23, 2026 @11:19AM (#47490729)

I think the real problem with using DSPy is that many of the problems people are trying to solve with LLMs (agents, chat) don't have an obvious path to evaluate. You have to really think carefully on how to build up a training and evaluation dataset that you can throw to DSPy to get it to optimize.

This takes a ton of upfront work and careful thinking. As soon as you move the goalposts of what you're trying to achieve you also have to update the training and evaluation dataset to cover that new use case.

This can actually get in the way of moving fast. Often teams are not trying to optimize their prompts but even trying to figure out what the set of questions and right answers should be!

[ Parent | on HN ]

Re:If DSPy is so great, why isn't anyone using it? by sbpayne on Monday March 23, 2026 @11:22AM (#47490767)

Yeah, I think Dspy often does not really show it's benefit until you have a good 'automated metric', which can be difficult to get to.

I think the unfortunate part is: the way it encourages you to structure your code is good for other reasons that might not be an 'acute' pain. And over time, it seems inevitable you'll end up building something that looks like it.

[ Parent | on HN ]

Re:If DSPy is so great, why isn't anyone using it? by memothon on Monday March 23, 2026 @11:24AM (#47490808)

Yeah I agree with this. I will try to use it in earnest on my next project.

That metric is the key piece. I don't know the right way to build an automated metric for a lot of the systems I want to build that will stand the test of time.

[ Parent | on HN ]

Re:If DSPy is so great, why isn't anyone using it? by sbpayne on Monday March 23, 2026 @12:12PM (#47491479)

To be clear: I don't know that I would recommend using it, exactly. I would just make sure you understand the lessons so you see how it best makes sense to apply to your project :)

[ Parent | on HN ]

Re:If DSPy is so great, why isn't anyone using it? by Eridrus on Monday March 23, 2026 @11:29PM (#47498311)

How are people building anything without evals?

Maybe I spent too much time in the ML mines, but it is somewhat inconceivable to iterate on a tricky problem without a eval set.

[ Parent | on HN ]