Paywalled.

This is a gift url/free of paywall

https://www.bloomberg.com/news/articles/2023-03-29/ai-chatgp...

Very nice of you, thank you.

Great pay for a truly horrible job!

Wow, Hacker News is a truly amazing community, if someone already tried the job and built a opinion that it's not a good job to have, even though the niche is very new.

Tell me more about how it was to work as a "prompt engineer"/"ai whisperer" and why you eventually quit? Was it a bit repetitive? Also, if you could say, where in the world was the company you worked for located? Did you like prompt engineering as a hobby before, got the job and then didn't like it, or why didn't it work out in the end?

To be fair, just because someone thinks a job is horrible doesn't mean they would quit.

I call BS. How are they gatekeeping without degrees?

Even internally at my company they won't let you touch the models without a PhD. Sure they'll let me be an "ML Engineer", but all that means is being a data jockey. It doesn't even pay any differently.

The jobs are to engineer prompts, not engineer an AI model. Although I think the pay is too high given the large number of people who could do the job.

Honestly I would expect candidates to use their AI Whispering skills to forge their resume and credentials. That's the whole point, isn't it?

Sound like a way to find people who "know how to ask the right questions". Nevermind using chatGPT, these might be the people you want running the company.

Prompt enginering is a domain where the supply will outstrip the demand, so the salary pricing will equalize itself incredibly quickly. Prompt engineering is also not a proprietary skill and it's very easy to teach someone non-ML or even nontechnical how to do it given a few examples, speaking from experience.

Unfortunately the rise of these types of jobs provides a strong incentive for casual users of LLMs to not share their prompts anymore. (hence why I'm open sourcing all my nonwork usage of ChatGPT and its prompts: https://news.ycombinator.com/item?id=35110998 )

I mostly agree with you. But I think we should be weary of statements like this:

> It's very easy to teach someone non-ML or even nontechnical how to do it given a few examples

Because a C programmer could have said the same thing about a Python job posting in 2004. Or an Assembly programmer could have said it about a job posting for a C programmer in 1975.

It's possible that prompt engineering will become the next higher level of abstraction above scripting languages, which means it will have its own ecosystem develop around it while introducing a new audience to the technical domain. That doesn't make it any less of a programming language, even if it appears that way now. Prompt engineering will have its own quirks and what makes a "good" prompt engineer may end up being somewhat distinct from what makes a "good" programmer. Maybe it will turn out that the best prompt engineers are the people most skilled with communicating precisely, which - to be fair - definitely includes programmers, but also opens the field to expert communicators from non-technical domains, like lawyers or editors.

All that said - paying anyone, non-technical or otherwise, $350k for talking to ChatGPT is obviously outrageous.

Most people are already used to prompt engineering thanks to years of using Google. They just aren't fully conscious of that connection yet, as "prompt engineering" sounds far more complex than it actually is.

Also worth noting, prompt engineering is only "difficult" now because the UI is effectively non existent, it's just a text box. This too will improve, and prompt engineering will get easier. It's imperative it becomes easier from the perspective of AI providers.

>That doesn't make it any less of a programming language

I don't see how you can believe that. Prompt engineering will never require learning an entirely new method of communication because the entire point is to communicate in languages you already know to create generations. Just because someone writes "image barn sunny trees" doesn't mean it's incomprehensible to those who know English, definitely not in the same way that "public static void main()" is incomprehensible to those who do understand English but not Java (or it's similar). You might think "well, prompt engineering can turn out the same way that Java example does", but that negates the entire functionality of AI and you're now just programming!

> Prompt engineering will never require learning an entirely new method of communication

No, but it's not like every person in the world is a super competent communicator even in their own language. But I agree the number of competent natural language communicators is an order of magnitude larger than the number of competent programmers. So I do agree with minimaxir that supply will rapidly outstrip demand, and it's unlikely that "prompt engineering" (which is a ridiculously stupid title, btw, can we still change that?) will ever be more highly paid than "real" programming (or whatever is left of real programming, at least).

Still, my point is mostly that we should be weary of making proclamations like this because it's not easy to predict how this field will develop, especially when you consider aspects like domain-specific expertise (e.g. lawyers will be in a better position than programmers to "prompt engineer" a legal contract), and the simultaneous replacement of existing jobs with AI. These factors will blend together to create a mix of incentives that makes it difficult to predict the end result.

I don't think the Assembly/C/Python analogy is perfect, but it's not far off from the truth - we see a new technology ushering in a new wave of people proficient in it, and the temptation is to wave it away as unlikely to affect the status quo, but reality never ends up being so simple.

On the contrary, I was not suggesting it is unlikely to affect the status quo, I think prompt engineering will become so default it's as expected as "can work on a team". Status quo will change, but not in the "direct replacement" theory people are so scared of.

I also don't believe that domain-specific expertise is the big exception here you believe it to be, as domain specific expertise is not gatekept in any efficient form. This will result in people being able to utilize such expertise just through AI - after all, you don't need to be an expert chef to know what a julienne technique is, just ask AI.

This is the new 'social media expert'.

Bracing for Robert Scoble to announce his new AI Whisperer company in his next sexual harassment victim blaming non-apology-apology.

https://web.archive.org/web/20171111033215/https://scobleize...

https://onemanandhisblog.com/2017/10/scoble-utterly-tone-dea...

https://www.theverge.com/2017/10/25/16547332/robert-scoble-s...

>The Verge‘s Adi Robertson sums it us thus: "But his latest defense puts forward an absurd definition of sexual harassment and effectively accuses women of reporting it to fit in with the cool crowd, while claiming he’s writing in “a spirit of healing.” There’s even a tasteless plug for his latest business venture. It’s one of the most disappointing responses we’ve seen to a sexual harassment complaint, which, after the past few weeks, is a fairly remarkable achievement."

I'm no fan of Scoble, but among AI hypemen on Twitter, he's far from influential.

Excellent description, just another gimmick

Except the fact that a vast number of people actually do this for a living and make good money. A good fraction (no comment on whether that’s larger than 0.5 or less) do something real and tangible and produce value to their clients too.

Importantly, the good ones, what they do, is not necessarily something everyone (especially people with the mind of an engineer) can do well even though in paper it’s a stupid job. I can see prompt engineering be the same.

The vast majority of people I’ve seen, the only original things they can think of asking chatGPT is to come up with workout plans or ask unoriginal questions about common societal issues. There’s nuance and imagination involved in figuring out how to effectively interact with this human like system to get it to do exactly what you want.

> it's very easy to teach someone non-ML or even nontechnical how to do it given a few examples

Sounds like we could train an AI to do it.

Just like any position, it's not hard to find X, but hard to find good X'ers. Same is true for developers, there are way more developers than demanded, but what's hard is finding good developers, not just any developer. Same is true for most professions that doesn't rely on certificates and alike (think "real" engineering, like engineering for construction and such).

Quoting myself from the post I linked above: as a machine learning engineer, I can attest that the only reasons machine learning engineers are good at prompt engineering are a) years of practice and b) a tendency to be pedantic assholes. But there are other professions who are even better at being pedantic assholes such as writers and lawyers, so there’s no need for someone with a specialized skillset to do it.

There is some granularity in terms of prompt engineering with ReACT/LangChain, which does require a machine learning background to make any sense out of it because it's very arcane. But that's not what these job openings are looking for, yet.

Requiring pedantry is also a system flaw. One would hope that AI assistants and delegates would understand more about what the human requesters want, and when to ask clarifying questions.

I’ve done dynamic prompt engineering with string concatenation

I’ve used this to create procedurally generated games, and noticed LLMs are able to dynamically generate NPCs and maintain state about those NPCs and their dialogue too

There is no way this is going to be a valuable skillset that humans do for salary. If the humans manage to become significant shareholders of the output or revenue stream, they’ll be fine, but for salary this is not valuable and computers will take prompt engineering too.

Senior Rockstar Prompt Juggler Ninja

I doubt this job actually exists at this salary and I suspect this is mainly a way to get some press coverage for the company doing the hiring

Having hired PR agents before, and also having been on the job market anytime in the last 6 months, I am very inclined to agree.

The way to read between the lines of these hot industries is to notice what salaries the third party recruiters are actually placing people at

Like the sister WaPo article hyping up this job title, also involving Anthropic (https://archive.is/pvwlx), this is still a super small market and only red-hot only because of Anthropic's job posting. It's like watching a penny stock have a huge price spike when there's one buyer and one seller.

In addition, I can't fathom paying $300k for skills less than a Ph.D or self-directed research. You'll be wanting to find emergent stuff like this and not just crafting prompts to get the right response: https://arxiv.org/abs/2201.11903 This is probably what prompt engineering should be about, not just crafting stuff for a particular output.

You can also get some freelance market rates here: https://promptbase.com/hire although the activity/reviews are quite low volume. Almost anyone can learn to do what these folks do, especially in stable diffusion land with all of the web-ui plugins.

It seems like even a PhD doesn't get you close to $300k at most research scientist type roles, with the exception of the big tech names.

Yup. At least with a physics PhD, it seems you make less than the software engineers sitting next to you even at a physics-centric company. Unfortunately, most people out of grad school are just happy to be making more and don’t exercise much negotiating power, suppressing wages for everyone.

You don't even need a masters to make $300K doing CRUD in big tech.

I don't have a bachelor's and I take home $300k

What a huge contrast lol. $335k salaries vs $3.99 per prompt.

$335K/yr is 335 $4 prompts per standard workday, about 1/minute, allowing for breaks.

How fast can you generate prompts?

I'm using AI to make my AI prompt.

Money Please!!!

Yep, just type: "teach me prompt engineering".

To me it sounds like an Accenture submarine. "Come and let us charge you for doing something you can do yourself". Maybe I'm just a cynical techie, but I don't think some guy who hasn't studied anything technical is going to provide any value in talking to the AI. Chances are you will do much better with domain knowledge, eg a dev asking it for code, or a lawyer asking it for legal precedents.

Linguistics (if you did it right) and Cognitive Science is deeply "technical" in the sense that you'll be knee deep in Tree Structure, Lexiostatistics, and Probability Theory to create datasets, reproducible grammars, and encode meaning.

Sure, but Phelps is a history grad.

True, but that's more an artifact of Accenture's hiring practices in the UK (concentrating on hiring from a handful of Russell Group Unis where Engineering doesn't have much of a reputation compared to humanities/social science. on a separate note, the only notable NLP/AI groups in the UK are at Cambridge, Oxford, Edinburgh, and Imperial College, and most of them filter into SWE).

The equivalent roles in the US will end up hiring people with a background in Linguistics, Cognitive Science, or Library and Information Sciences. This is seen in Anthropic's job listing for example.

You're on to something here.

I really believe that in the long run, it will be the tech oriented domain experts who are the valuable ones. That's really who we've found we need at nearly every implementation step of our AI strategy. You need to know what data to get in your target domain. How to interpret that data. Some method of scoring that data. And all that's just table stakes. It goes on and on. Someone with a tech background is just not going to know about stenosis calculations the way a cardiologist will. Or legal precedents like a lawyer does. Or compressive strength values like a civil engineer does. And so on and so forth.

AI generalists are also enormously valuable, but only to the Google, Microsofts, and OpenAIs of the world. Among those competing to make generalist AIs, it will be winner take all I'm afraid. Majority of the jobs created for generalist AI guys will be at big tech cos. The vast majority of the jobs generated in AI outside of big tech, will rely heavily on specialized knowledge of the domain experts.

I even think colleges might want to consider adjusting the educational curriculum so that some cs knowledge is taught more like writing class. It's just something professionals will need to be familiar with and know how to do in the future. But not all that valuable in and of itself unless you can be one of the top guys.

Someone who only knows CS? no other specialized knowledge?

And they don't have the math chops of the OpenAI guys?

Sorry, but you're just not going to be all that valuable in the future. Salaries for those kinds of people will start a slow decline for sure.

Probably the job posting was just to provide legal cover for some spectacularly indulgent act of nepotism, like hiring the CEO's otherwise useless frat boy son after he got kicked out of school for cocaine and sexual abuse. I'm surprised Truth Social hasn't hired Don Jr. to write conspiracy theories and press releases by whispering to ChatGPT3 yet.

In my experience this is spot on. We hired the COOs son and paid him $200k+ various stipends.

"Prompt engineering" is a made up thing. There's not some magic William Gibson prompt cowboy who through some innate intuition can master prompts the way nobody else can.

It's a deterministic model. There's plenty of interesting explainability work, or however you want to frame it, to be done understanding the connection between different prompts and outputs. But it's clearly in the province of real ML researchers who have experience on the area, and will approach it scientifically. 335k for someone good at that is not unusual at all, they have lots of options.

Prompt engineering is more prompt chaining which is actively being researched so it is a thing. https://github.com/hwchase17/langchain

Point is, there's some black box, lay-version of playing with prompts, and there's a real study of how the prompt affects output by people who understand the workings of the model. Presumably they're hiring for the latter and not some kid with intuition, and therefore the headline is sensational because the story is that a job for a highly educated and specialized scientist pays well.

> It's a deterministic model

What is? Aren't large language models very specifically not deterministic? It's impossible to reliably predict their response to a given prompt until you actually run the prompt, even with the temperature parameter set to zero (and often temperature is higher than zero in practical deployments).

It's all well and fine until somebody figures out you can use ChatGPT to write ChatGPT prompts.

The pay range for this job is more indicative of Anthropic having massive salaries across the board for most of their positions. This is partially also because they're wanting people on-site in SF. Their senior software engineer role is listed $270-$445k a year. Recruiter is up to $255k a year. Design lead up to $380k a year. TPM up to $445k.

More accurate headline: Solutions architect role pays 30% more than recruiter role, but still $100k less than a software developer

Now we need next gen GPT to automate "prompt engineering" itself. I guess this means being able to

1- Better "understand" your personal way/style of communicating and translate it to the model (maybe by being trained on your personal chats/emails/notes/docs?)

2- Ask you enough questions (like a consultant) to infer a clearer understanding of what you mean (by resolving unsaid assumptions, deciding between sub-choices...)

Anthropic position aside - can anyone recommend any resources for learning more about the actual work involved with prompt engineering?

Revealing; this translates to "we need to hire a $335k employee to figure out how to use our product." Associating anything AI with money inflates the bubble.