For example right now we have AIs writing CVs and cover letters that are read by AIs, we're "optimizing" things that shouldn't exist anymore but we're too deep in the cycle to even notice it
Some communication is intended to inform, some is intended to persuade... And often, the speaker doesn't even really know which they are dealing with.
The best way to get people to interact with you a certain way is to reward them for interacting with you that way, and you likely don't. Not because that's a failure of you but it's just how people (all of us) act.
The world isn't a co-operative multi-tasking OS. We're more like the human body: a multi-agent conglomerate whose component organisms communicate via signals that are filtered and processed. Sometimes a pathogen mimics signals well enough. Other times we detect it. But the system wouldn't work if every participant trusted the counterparty.
Ability to interpret/summarize is limited by context size and time.
Document dumping is the social DoS.
So now you have 2 machines talking to each other in text-to-speech English over audio, then having to do speech-to-text on each others audio streams, and then natural language processing to understand the message etc.
When all we needed instead was a simple API endpoint over the internet for the business where a machine could talk to the other machine with just a few bytes of information to convey the same information and process the same transaction.
Instead we are wasting bandwidth and tons of cpu cycles and electricity making the machines emulate a human analog conversation.
Maybe in the not so distant future, the AI agents can have a shibboleth which cuts the human gobbledygook short, and reverts to speaking in dialup tones.
I disagree here.
But what I found LLMs really useful for is summarizing topics like: "Suppose I want to do XYZ, which steps should I take?" or "I want to write about ABC, how would you structure the article?"
The content is not very good if you look at it in detail, but LLMs provide good overviews and gave me ideas on parts of a topic I overlooked.
More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
> More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
The same can be said of physical books, and most content should be written more like that in books than what the internet is currently selecting for.
Incidentally, this makes a fine tuned version perfect for the government, which produces a shit load of low-effort documents, but today employs a bunch of morons to write them.
If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
I appreciate the rest of your post though and I think you make a great point about AI being very effective for certain types of work
Some of them could be in the beginning part of their government career attending school. Did you run a survey to ask these people what they do with their lives?
> If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
Granted, we are judging LLMs based on highly generalized training sets. What if a GPT was fine-tuned on all of the writing and speeches of figures whom are considered the most persuasive?
Government reports continue to balloon in size, because of technological advancements (especially copy/paste).
In the age of typewriters, anything not important, or duplicate, was cut. With word processors, document sizes unnecessarily grew.
All things considered, an awkward sentence or two isn't really that much of a big deal compared to an incorrect assertion that was missed in review.
I also don't think GPT-4 is necessarily that brilliant at writing copy, but I don't really know how good/bad US government officials are so I can't really compare, but in other parts of the world I haven't really that much of an issue with it.
For the boilerplate- well sure, but I could do that with existing tools, no AI needed. Custom communications I'm sure AI can handle the very basics. but once a human replies and consideration is needed for special cases- can you imagine ChatGPT counseling you on a healthcare decision? I can't.
I can't imagine ChatGPT doing much more than being an autoattendant until a human can reply fully, except the message humans can send to the agency will be customized and drafted according to a standard intake process.
The question is, do you want an AI writing the brief and doing the research that is going to be given to an administrator who is going to decide if you are denied medical care in an emergency?
A disturbingly high proportion of US adults (at least in the US—I don't know how this looks in other countries) barely count as literate. Perhaps half[1]. Most of the rest aren't a ton better, and can't write worth a damn.
I think "on average" understates the case: at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not. Most of that remaining 10% would also see improvements from ChatGPT, except for writing they've put a great deal of effort into. It's very good at cleaning up writing, and/or tuning it to be better-suited to some purpose than the original was.
In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT and having it summarize it concisely, in very simple terms.
The world makes a lot more sense, to those who grew up taking to the written word as naturally as a duck to water, when we appreciate that the world runs on the written word, yet most people are terrible readers and worse writers.
[1] "According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level." And, nb, if you're a good reader and you're thinking "6th grade level, that's not so bad", your idea of what a "6th-grade level" looks like, is probably skewed way higher than what they mean.
Bullet list -> AI text generator -> AI text summarizer -> Bullet list
Perhaps OpenAI can optimize the process by storing the original prompt and just giving it back at the last phase.
this is quite saddening.
IIRC the high-school-graduate equivalent adult literacy rate from similar studies—as in, possessing reading skills that we hope a high school graduate would have, which basically equates to being able to read two moderate-length, moderate-complexity texts and understand them well enough to synthesize a description of their positions, and how they agree and differ—isn't much more than 20%. Four out of five people you encounter are going through life constantly struggling to understand WTF is happening or what they're supposed to do and getting by on a lot of guesswork, while all continually mis-communicating with one another. Really explains a lot.
To continue the entirety of the quote which makes it not seem quite as dammning:
"Literacy in the United States was categorized by the National Center for Education Statistics into different literacy levels, with 92% of American adults having at least "Level 1" literacy in 2019.[1] According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level.[2]
In many nations, the ability to read a simple sentence suffices as literacy, and was the previous standard for the U.S. The definition of literacy has changed greatly; the term is presently defined as the ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential.[3]"
I'd expect writing samples to get shorter, not longer, when being improved.
LLMs, in my opinion, are on average better at writing copy than humans. Or rather they manage to do so more consistently. Thus, even if humans still need to provide supervision, the clarity and conciseness of government documents may be improved overall.