As an experienced LLM user, I don't use generative LLMs often

Re:As an experienced LLM user, I don't use generative LLMs o... by tstrimple on Tuesday May 06, 2025 @02:59AM (#43902454)

I think “don’t use the context window” might be too simple. It can be incredibly useful. But avoid getting in a context window loop. When iterations stop showing useful progress toward the goal, it’s time to abandon the context. LLMs tend to circle back to the same dead end solution path at some point. It also helps to jump between LLMs to get a feel for how they perform on different problem spaces.

[ Parent | on HN ]

Re:As an experienced LLM user, I don't use generative LLMs o... by emporas on Tuesday May 06, 2025 @03:18AM (#43902546)

Depends on the use case. For programming where every small detail might have huge implications, 98% accuracy vs 99.99% is a ginormous difference.

Other tasks can be more forgiving, like writing which i do all the time, then i load 3000 tokens in the context window pretty frequently. Small details in the accuracy do not matter so much for most people, for everyday casual tasks like rewriting text, summarizing etc.

In general, be wary of how much context you load into the chat, performance degrades faster than you can imagine. Ok, the aphorism i started with, was a little simplistic.

[ Parent | on HN ]

Re:As an experienced LLM user, I don't use generative LLMs o... by tstrimple on Tuesday May 06, 2025 @03:38AM (#43902636)

Oh sure. Context windows have been less useful for me on programming tasks than other things. When working iteratively against a CSV file for instance, it can be very useful. I’ve used something very similar to the following before:

“Okay, now add another column which is the price of the VM based on current Azure costs and the CPU and Memory requirements listed.”

“This seems to only use a few Azure VM SKU. Use the full list.”

“Can you remove the burstable SKU?”

Though I will say simple error fixes with context windows with programming issues are resolved fine. On more than one occasion when I copy and paste an incorrect solution. Providing the LLM with the error is enough to fix the problem. But if it goes beyond that, it’s best to abandon the context.

[ Parent | on HN ]