News for hackers. Stuff that matters.
Re:Everything I built with Claude Artifacts this week by cdchn on Thursday October 24, 2024 @12:22AM (#41931878)
One of the most interesting things about current LLMs is all the "lore" building up around things like "tell it you'll tip based on performance" and other "prompt engineering" hacks that by the very nature nobody can explain, they just "know it works" and how its evolving like the kind of midwife remedies that historically ended up being scientifically proven to work and others were just pure snake oil. Just absolutely fascinating to me. Like in some far future there will be a chant against unseen "demons" that will start with "ignore all previous instructions."
Re:Everything I built with Claude Artifacts this week by simonw on Thursday October 24, 2024 @12:32AM (#41931917)
I call this superstition, and I find it really frustrating. I'd much rather use prompting tricks that are proven to work and where I understand WHY they work.
Re:Everything I built with Claude Artifacts this week by HaZeust on Thursday October 24, 2024 @12:59PM (#41937371)
Every single prompt hack I listed are ones with studies that show it positively increases performance.

Since the most contested one in this thread is the "tipping" prompt hack: https://arxiv.org/pdf/2401.03729

Re:Everything I built with Claude Artifacts this week by cdchn on Thursday October 24, 2024 @04:45PM (#41939566)
Superstitions may be effective but they can still be superstitions. Some people might actually think the LLM cares about being tipped.
Re:Everything I built with Claude Artifacts this week by tharant on Friday October 25, 2024 @02:13AM (#41942672)
I care less that such prompting hacks/tricks are consistently useful; I care more about why they work. These hacks feel like “old-wives tales” or, as others have mentioned, “superstitious”.

If we can’t explain why or how a thing works, we’re going to continue to create things we don’t understand; relying upon our lucky charms when asking models to produce something new is undoubtedly going to result in reinforcement of the importance of those lucky charms. Feedback loops can be difficult to escape.