News for hackers. Stuff that matters.
Re:AI Prompt Engineering Is Dead by dpflan on Wednesday March 06, 2024 @10:54AM (#39617243)
If you have been hired as a "Prompt Engineer", can you please explain your work? Are you just finding better ways to communicate clearly in human language to the current model you are interacting with? Or are you doing statistical analyses of prompts, etc? It is hard for me not think a Prompt Engineer as most basic is someone comfortable and knowledgeable about how to express a concept in human language to an LLM, which to me, really sounds like at minimum: clearly communicate with little ambiguity.
Re:AI Prompt Engineering Is Dead by binarymax on Wednesday March 06, 2024 @10:58AM (#39617302)
I'm not a full time "prompt engineer" but it's a small role in the work that I do for my business.

Prompt engineering is kinda like a basic form of data science. You have a dataset and some manually labelled results, and you hypothesize on prompts and try to improve on some metric. You'd be surprised how the tiniest of changes will alter a result. Thinks like a hyphen, semicolon, or capitalization can sway the metric. It's very finicky and very annoying.

Re:AI Prompt Engineering Is Dead by msp26 on Wednesday March 06, 2024 @11:13AM (#39617431)
Tokenization errors are also really, really annoying to deal with.

e.g. why does the JSON output have silly whitespace/quotation sometimes? Obviously it's because the first token the model output was `{` and not `{"` like it should be. Obviously.

Re:AI Prompt Engineering Is Dead by binarymax on Wednesday March 06, 2024 @11:27AM (#39617585)
Totally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with strict JSON because of the error rate - and instead opt for HTML which is more forgiving when parsing.
Re:AI Prompt Engineering Is Dead by msp26 on Wednesday March 06, 2024 @11:49AM (#39617850)
nono, the same thing happens with OpenAI's JSON mode and function calling. It does output parsable JSON very reliably but it comes out mangled with a bunch of whitespace sometimes.

GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.

Re:AI Prompt Engineering Is Dead by hanniabu on Wednesday March 06, 2024 @10:57AM (#39617288)
It's not as simple as being descriptive, but what you're of descriptive and knowing the terms to use.

You also need to know the tools to use and there's a lot of settings you play with that are hidden from the user in mainstream apps.