News for hackers. Stuff that matters.
Re:OpenAI co-founder John Schulman says he will leave and jo... by Workaccount2 on Tuesday August 06, 2024 @10:06AM (#41170996)
I don't mean this as a slight, just an observation I have seen many times - people who struggle with utility from SOTA LLM's tend to not have spent enough time with them to feel out good prompting. In the same way that there is a skill for googling information, there is a skill for teasing consistent good responses from LLM's.
Re:OpenAI co-founder John Schulman says he will leave and jo... by dartos on Tuesday August 06, 2024 @11:05AM (#41171578)
Why spend my time teasing and coaxing information out of a system which absolutely does make up nonsense when I can just read the manual?

I spent 2023 developing LLM powered chatbots with people who, purportedly, were very good at prompting, but never saw any better output than what I got for the tasks I’m interested in.

I think the “you need to get good at prompting” idea is very shallow. There’s really not much to learn about prompting. It’s all hacks and anecdotes which could change drastically from model to model.

None of which, from what I’ve seen, makes up for the limitations of LLM no matter how many times I try adding “your job depends on Formatting this correctly “ or reordering my prompt so that more relevant information is later, etc

Prompt engineering has improved RAG pipelines I’ve worked on though, just not anything in the realm of comprehension or planning of any amount of real complexity.

Re:OpenAI co-founder John Schulman says he will leave and jo... by danielbln on Tuesday August 06, 2024 @10:21AM (#41171112)
People also continue to use them as knowledge databases, despite that not being where they shine. Give enough context into the model (descriptions, code, documentation, ideas, examples) and have a dialog, that's where these strong LLMs really shine.
Re:OpenAI co-founder John Schulman says he will leave and jo... by dartos on Tuesday August 06, 2024 @11:12AM (#41171656)
Summarizing, doc qa, and unstructured text ingestion are the killer features I’ve seen.

The 3rd one still being quite involved, but leaps and bounds easier than 5 years ago.