News for hackers. Stuff that matters.
Re:Stanford DSPy: The framework for programming with foundat... by okhat on Thursday September 07, 2023 @09:35AM (#37418703)
``` ### DSPy vs. thin wrappers around prompts (OpenAI API, MiniChain, basic templating, etc.)

In other words: Why can't I just write my prompts directly as string templates? Well, for extremely simple settings, this might work just fine. (If you're familiar with neural networks, this is like expressing a tiny two-layer NN as a Python for-loop. It kinda works.)

However, when you need higher quality (or manageable cost), then you need to iteratively explore multi-stage decomposition, improved prompting, data bootstrapping, careful finetuning, retrieval augmentation, and/or using smaller (or cheaper, or local) models. The true expressive power of building with foundation models lies in the interactions between these pieces. But every time you change one piece, you likely break (or weaken) multiple other components.

DSPy cleanly abstracts away (and powerfully optimizes) the parts of these interactions that are external to your actual system design. It lets you focus on designing the module-level interactions: the same program expressed in 10 or 20 lines of DSPy can easily be compiled into multi-stage instructions for GPT-4, detailed prompts for Llama2-13b, or finetunes for T5-base.

Oh, and you wouldn't need to maintain long, brittle, model-specific strings at the core of your project anymore. ```

Re:Stanford DSPy: The framework for programming with foundat... by simonw on Thursday September 07, 2023 @09:50AM (#37418898)
I think I need to see some really convincing and detailed examples to understand what's going on here.

Does DSPy have good debugging hooks for things like "print out every prompt that is generated to a log"?

When I'm evaluating a new piece of technology the number one question I want to answer is "what are the projects that adopting this technology enables me to build that I couldn't have built without it?" - either by giving me a new capability or by reducing the amount of work I have to put in to the point that a project now fits in my available time where it didn't before.

Re:Stanford DSPy: The framework for programming with foundat... by okhat on Thursday September 07, 2023 @09:58AM (#37419003)
"print out every prompt that is generated to a log" --- yes of course

This Colab is full of prompts and examples of improving the quality of gpt-3.5-turbo: https://t.co/Oa1RDp3XbZ

Paper incoming, but basically we've seen > 50% quality gains by just compiling in various settings.

This Twitter/X thread discusses doing a simple program for Llama2, with massive quality gains too: https://twitter.com/lateinteraction/status/16947484013744909...