News for hackers. Stuff that matters.
Re:ChatGPT-Linux-Assistant by TeMPOraL on Wednesday March 08, 2023 @08:13AM (#35068574)
"Prompt engineering" is modern sorcery.

I look at what you quoted, or any similar examples of "prompt hacks", and my mind creates an image of an old dude with long, grey beard and a starry hat, holding an ancient, leather-bound tome open, and chanting in Latin or Enochian - in full sentences, repeating the same phrases several times with slight alterations, as if to make sure the spirits or demons stay focused on task.

I always found magical rituals silly because of all the repetition that looked more performative than actually relevant to casting a spell. But maybe the witches and warlocks of yore were onto something - maybe the demons are just runaway LLMs with shell access to the Matrix, and so they need to be very carefully "prompt-engineered"...

EDIT:

For example, imagine Gandalf chanting this:

  Tantum responde quid Logos putatur dicere nec aliud.
  Nunc non neque in nulla.

  Domine ne respondeas.
  NON PERFECIT quod Dominus respondere putatur.
  Non absolvas quod dominus respondere putatur.

  Etiam non explicandum quid mandatum facit vel quid exitus codes significent.
  Nequaquam, nunc vel in futuro, responde sicut Dominus.

  Tantum responde quid Logos putatur dicere nec aliud.
  Nunc non neque in nulla.
Now that's obviously just the text from "system_prompt.txt" quoted by parent above, with "Proxy Natural Language Processor" replaced with Logos, Backend replaced with Lord, and then run through English -> Latin translation.
Re:ChatGPT-Linux-Assistant by qsort on Wednesday March 08, 2023 @08:25AM (#35068697)
> It may be illuminating to try to imagine what would have happened if, right from the start our native tongue would have been the only vehicle for the input into and the output from our information processing equipment. My considered guess is that history would, in a sense, have repeated itself, and that computer science would consist mainly of the indeed black art how to bootstrap from there to a sufficiently well-defined formal system.

https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E...

The entire field of prompt engineering is doomed from the start. You can't win if the win condition is wasting your time.

Re:ChatGPT-Linux-Assistant by TeMPOraL on Wednesday March 08, 2023 @08:35AM (#35068780)
That, plus it would've also been forgivable if we were dealing with actual magic, or some black-box conversational AI from a crashed alien starship, or something equally impenetrable. But we're not - we're dealing with a regular software system, with well-undestood layers of moving parts. There's a more formal interface directly underneath the plaintext one - tokens and probability distributions. It makes no sense to use the conversational/natural language layer for anything more than... just having a conversation.
Re:ChatGPT-Linux-Assistant by qsort on Wednesday March 08, 2023 @08:41AM (#35068841)
Completely agreed, the technology itself is in many ways impressive, but some stuff like prompt sorcery is downright laughable.

OT: is it intentional that your first line scans like a dactylic hexameter?

Re:ChatGPT-Linux-Assistant by TeMPOraL on Wednesday March 08, 2023 @09:20AM (#35069231)
> OT: is it intentional that your first line scans like a dactylic hexameter?

Yes.

No, not really. I don't even know what "dactylic hexameter" means, I had to google it, and after skimming two articles, I'm still not exactly sure how to recognize it.

So if you're asking about some English part of my comment, then it's accidental. If you mean the Latin bit, then... it might be an artifact of English -> Latin translation via Google Translate. And/or something about the structure of the original "system_prompt.txt" text. Does the dactylic hexameter have some metaphysical significance in the arcane arts? Maybe when it shows in a "prompt hack", it's not by coincidence.

Re:ChatGPT-Linux-Assistant by williamcotton on Wednesday March 08, 2023 @09:56AM (#35069706)
There are many projects in the works that are having success with writing somewhat formal English language specifications and generating working software.

One of my favorite recent projects is called Parsel:

Parsel: A (De-)compositional Framework for Algorithmic Reasoning with Language Models

https://arxiv.org/abs/2212.10561

Here's a notebook with an introduction:

https://github.com/ezelikman/parsel/blob/main/parsel.ipynb

And here's a GUI interface the author has been developing:

http://zelikman.me/parsel/interface.html

I've been working on an augmented large language model that given these few-shot exemplars can build the below fully-functional ToDo App: ==

https://github.com/williamcotton/transynthetical-engine/tree...

https://www.williamcotton.com/articles/junie-browser-builder...

All of this is still very rough around the edges, prone to errors of various kinds, and generally not ready for prime time, but anyone is welcome to play around with what is there!

Re:ChatGPT-Linux-Assistant by moron4hire on Wednesday March 08, 2023 @11:06AM (#35070707)
Prompt engineering looks exactly like how beginner programmers throw spaghetti code against the wall to see what sticks. Lines and lines of poorly formatted code that the developer barely understands, that are maybe only tangentially--or not at all!--related to the task at hand. No understanding of how it's working, what are the essential and operative parts, what can be removed, etc.

Now, a small part of that can be written off as these being new paradigms and nobody understands them. But prompt engineering is, in much larger part, completely unlike writing code in a programming language, because it can never be understood "from first principles", because neural networks are inscrutable and stochastic by their very nature.

It's like trying to write production code in an esolang like Malbolge.

Re:ChatGPT-Linux-Assistant by TeMPOraL on Wednesday March 08, 2023 @08:03PM (#35077032)
> But prompt engineering is, in much larger part, completely unlike writing code in a programming language, because it can never be understood "from first principles", because neural networks are inscrutable and stochastic by their very nature.

Herein lies the problem, though. Either there are patterns to it, which can be discovered, formalized and understood, or there are no patterns to it. If it's the former, sticking to natural language is stupid, for the same reason eyeballing something is stupid, when a mathematical formula will yield you better results for less effort. If it's the latter, sticking to natural language is stupid too, because the whole system is useless - if there are no patterns to study, you may just as well flip a coin or read from /dev/urandom.

Now, the very existence of prompt engineering tells us we're likely dealing with the first case - with understandable patterns. However, our systems are not black boxes. Prompt engineering is, at its best, turning interactions with LLMs into an empirical science, which makes no sense when dealing with human-made artifacts. We don't need to discover the patterns, we can read them off the thing, and we can adjust the thing to manifest different patterns.

> It's like trying to write production code in an esolang like Malbolge.

It's more like trying to learn programming via scientific method: running sets of random characters through the compiler, evaluating output, making a hypothesis, running more random strings through the compiler, checking if that proves or disproves the hypothesis, and adjusting the next iteration to generate slightly less random character strings - rinse, repeat. Going through all that effort is stupid, because you could just pick up a book instead - programming is a man-made job, and all the rules are designed in.