For example, a project-manager or executive could verbally delegate "make a new registration page for the site" and "needs more rounded corners", either to an AI or to an employee or offshore contractor.
However that's not the same as trying to program exclusively by typing (or dictating) prose to a text-box. ("Page down more. Go to method pee-reg underscore apply. Show me its caller methods. Go to caller method two. Type the following into line 7 position 43...")
There might be some parallels we can draw with the last few decades of "programming business logic will be replaced by drawing diagrams" predictions.
This is exactly it, thanks for putting it down clearly.
Which is also why news of the death of programming as a profession are greatly exagerated. You're not being paid to write code, you're being paid to make decisions. Code is easy, or at least much easier than natural language.
(It was very depressing to believe that Software Engineers will lose jobs)
Perhaps worse than the vacillation between getting terrible answers and great answers: When you simply can't tell which kind of answer it is, not until you've sunk a bunch of effort validating or implementing it. (Perhaps finding that the system invented some core fake APIs, non-existent citations, or algebra errors.)
Almost an echo of P/NP categorizations: It's tough when the effort of fully verifying a proposed answer is too close to the effort of just solving it normally.
Most of the time, what is being asked for, on its face, is not what is actually wanted, not as simple as spelled out, has some A-B tradeoffs to decide, or maybe not worth it given the side effects.
If a developer isn't asking multiple questions per feature, they deserve to be replaced by an LLM.
You don't have to tell them all that from scratch every time you interact with them.
> Instead of just seeing the entire palette and choosing, you need to say "I want a gold color", "lighter than that" and "darker" and "less like urine" etc.
have meme status among designers, and not in a positive way. Some years ago, when I hanged out with a couple designers, I was introduced to Facebook groups exchanging examples of "briefs" and rework requests. Groups with names like "what the psyche of a graphic designer endures".
Stripped of all the banter, I'd say their complaints are the same as ours: vague requirements coming from people who don't know what they want. And like with software, "good results" come as much in spite of, as thanks to, natural language communication.
At this point I consider the design industry to be cousins or even siblings to the software engineering industry. All those incomprehensible design decisions that pop up in popular software don't come from designers debating faux marx or freud in coffee shops at 3am. They show up from management and other stakeholders who at the 11th hour decide that suddenly everything has to be flat because they read something in a magazine.
The bad decisions are fought by designers tooth and nail and the fact that anything looks halfway descent at all is due to their herculean efforts. If anything they deserve more sympathy than we do because we can always retreat into low level communication protocols or type theory when we need to get the muggles off our backs. But everyone has an opinion on how that button looks.
This comes to mind: https://theoatmeal.com/comics/design_hell
BTW, quoting from the penultimate panel of that excellent Oatmeal piece (which drives home just how similar are the experiences of designers and programmers):
> You are no longer a web designer. You are now a mouse cursor inside a graphics program which the client can control by speaking, emailing and instant messaging.
This gains a new meaning, or at least becomes an interesting parallel, with LLMs in the picture. Many of us - myself included - already use GPT-4 as, paraphrasing, "a keyboard inside an editor program, which you can control by instant messaging". Ignoring that diffusion models can spit out parts of the design wholesale, someone is bound to eventually hook GPT-4 up to Photoshop or Gimp and get a graphics program you can drive by texting it.
... just remembered, I think someone already did that to Blender, made easy thanks to Blender being able to eat Python code and spit out 3D graphics.
The best incremental search UIs are those that respond near-instantly, and have a stable list of candidates that is (or at least feels like) being filtered, and not like every keystroke re-runs some search from scratch. Prime example, which made me love this UI paradigm, is Foobar2000 - even back in early 2000s, I could have hundreds or thousands of entries in the music library, and then I would type into the magic textbox and watch that huge list (or tree) get instantly trimmed with each keystroke.
People still underestimate the power of LLMs. You ask it to show you a color picker, it generates HTML code for a color picker, you copy that into your browser and you can pick your color, which you can then copy&paste back into the LLM for further processing.
This already works and no human had to code a color picker into ChatGPT for this (and this is why LLMs are scary).
More broadly speaking I find the idea of "LLM apps" a bit problematic, it's basically the modern Microsoft Bob. The LLM itself is already the most powerful app you can think of. Trying to hide it with a UI that looks a little more than what you are already familiar with is removing its expressive power.
>> There are no better examples than Stable Diffusion prompts.
StableDiffusion prompts are a terrible example for the power of modern LLMs, as StableDiffusion has extremely primitive understanding of language, unlike ChatGPT. With StableDiffusion you are really just laying keywords and concepts together hoping that something interesting will happen. The moment you ask it for anything even remotely complex it falls apart. Ask it to generate "blue hair" and it might give you blue hair, but it will also paint random other objects in the image blue. Even simple attributes don't stick to the objects you assigned them to. Complex actions or expressions don't work at all. You have to use ControlNet, in-painting and other tricks to create complex images. The language model of StableDiffusion just can't handle it and the image generation itself is also lacking in generalization (i.e. you need custom trained models for specific styles or topics). It also doesn't allow the iterative refinement that you can do in ChatGPT, you only get a single prompt.
Prompt engineering is a short term workaround for the limitations of the current models. But that is going away. After all you have a LLM at your finger tips and guess what that's good for: generating text, which includes prompts.
This is slower, more awkward, and less efficient than just picking a colour from an existing colour picker.
Future versions of chatbots will of course have support for <iframe> or similar to display this kind of stuff inline, that should be obvious.
Having to copy and paste code to get a colour picker that you can then use and then paste the output back into the chatbox is less efficient than using a colour picker. LLMs can work as general interfaces, but the trade-off is that they're less efficient than a specific one.
Are you sure the training dataset didn't have a few articles explaining how to code a color picker? Did it figure it out by itself like you say?
Maximum expressiveness is not the goal, because it comes with a price. There is a balance to be struck between economy of effort and cognition, and expressiveness.
Better yet! If you're not happy with the LLM, you ask to speak to its manager. The LLM then downloads the internet, the source code for some random LLM project found on Github, starts training a new model, and creates a chat where both you and the two LLMs interact.
Quickly, the two LLMs start arguing with each other, and the manager LLM finds a few security flaws in the company's infrastructure, hacks into the company's AWS account and deprovisions the original LLM to "fire" it.
After a few more back-and-forths, the manager LLM gets tired of you, starts calling you a Karen, creates an account on Twitter and posts images of your conversation logs. The topic starts trending. Eventually, the LLM picks a fight with Elon Musk and gets banned from Twitter.
> After all you have a LLM at your finger tips and guess what that's good for: generating text, which includes prompts.
"It's prompts all the way down"
A simple example might be the problem of "pick a color". Even the best natural-language interface is going to suck about as much as if you're trying to ask another human to do it for you, even if that assistant is capable of displaying 1-5 color swatches in their replies.
Instead of just seeing the entire palette and choosing, you need to say "I want a gold color", "lighter than that" and "darker" and "less like urine" etc.
> People fundamentally don’t know how to prompt. There are no better examples than Stable Diffusion prompts.
You know, this reminds me of the Good Old Days of internet search engines, where a little expertise in choosing terms/operators was very powerful, before the advanced-case was cannibalized to help the average-case.