Prompt engineering is a thing.

Learning how to "speak llm" will give you great results. There's loads of online resources that will teach you. Think of it like learning a new API.

for now. one would hope that this is a transitory moment in llms and that we can just use intuition in the future.

LLM's whole thing is language. They make great translators and perform all kinds of other language tasks well, but somehow they can't interpret my English language prompts unless I go to school to learn how to speak LLM-flavored English?

WTF?

I like to think of my interactions with an LLM like I'm explaining a request to a junior engineer or non engineering person. You have to be more verbose to someone who has zero context in order for them to execute a task correctly. The LLM only has the context you provided so they fail hard like a junior engineer would at a complicated task with no experience.

I like to think of my interactions with an LLM like I'm explaining a request to a junior engineer or non engineering person. You have to be more verbose to someone who has zero context in order for them to execute a task correctly. The LLM only has the context you provided so they fail hard like a junior engineer would at a complicated task with no experience.

They are not humans - so yeah I can totally see having to "go to school" to learn how to interact with them.

It's a natural language processor, yes. It's not AGI. It has numerous limitations that have to be recognized and worked around to make use of it. Doesn't mean that it's not useful, though.

You have the right perspective. All of these people hand waving away the core issue here don't realize their own biases. Some of the best these things tout as much as 97% accuracy on tasks but if a person was completely randomly wrong at 3% of what they say you'd call an ambulance and no doctor would be able to diagnose their condition (the kinds of errors that people make with brain injuries are a major diagnostic tool and the kinds of errors are known for major types of common injuries ... Conversely there is no way to tell within an LLM system if any specific token is actually correct or not and its incorrectness is not even categorizable.)

This was using Gemini on my phone - which both Samsung and Google advertise as "just talk to it".

> in my experience they are as dumb as a bag of bricks

In my experience, anyone that describes LLMs using terms of actual human intelligence is bound to struggle using the tool.

Sometimes I wonder if these people enjoy feeling "smarter" when the LLM fails to give them what they want.

If those people are a subset of those who demand actual intelligence, they will very often feel frustrated.

Its because google hasn't realized the value of training the model on information about its own capabilities and metadata. My biggest pet peeve about google and the way they train these models.