LLMs, in my opinion, are on average better at writing copy than humans. Or rather they manage to do so more consistently. Thus, even if humans still need to provide supervision, the clarity and conciseness of government documents may be improved overall.
For example right now we have AIs writing CVs and cover letters that are read by AIs, we're "optimizing" things that shouldn't exist anymore but we're too deep in the cycle to even notice it
Some communication is intended to inform, some is intended to persuade... And often, the speaker doesn't even really know which they are dealing with.
The best way to get people to interact with you a certain way is to reward them for interacting with you that way, and you likely don't. Not because that's a failure of you but it's just how people (all of us) act.
The world isn't a co-operative multi-tasking OS. We're more like the human body: a multi-agent conglomerate whose component organisms communicate via signals that are filtered and processed. Sometimes a pathogen mimics signals well enough. Other times we detect it. But the system wouldn't work if every participant trusted the counterparty.
Ability to interpret/summarize is limited by context size and time.
Document dumping is the social DoS.
So now you have 2 machines talking to each other in text-to-speech English over audio, then having to do speech-to-text on each others audio streams, and then natural language processing to understand the message etc.
When all we needed instead was a simple API endpoint over the internet for the business where a machine could talk to the other machine with just a few bytes of information to convey the same information and process the same transaction.
Instead we are wasting bandwidth and tons of cpu cycles and electricity making the machines emulate a human analog conversation.
Maybe in the not so distant future, the AI agents can have a shibboleth which cuts the human gobbledygook short, and reverts to speaking in dialup tones.
I disagree here.
But what I found LLMs really useful for is summarizing topics like: "Suppose I want to do XYZ, which steps should I take?" or "I want to write about ABC, how would you structure the article?"
The content is not very good if you look at it in detail, but LLMs provide good overviews and gave me ideas on parts of a topic I overlooked.
More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
> More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
The same can be said of physical books, and most content should be written more like that in books than what the internet is currently selecting for.
Incidentally, this makes a fine tuned version perfect for the government, which produces a shit load of low-effort documents, but today employs a bunch of morons to write them.
If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
I appreciate the rest of your post though and I think you make a great point about AI being very effective for certain types of work
Some of them could be in the beginning part of their government career attending school. Did you run a survey to ask these people what they do with their lives?
> If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
Granted, we are judging LLMs based on highly generalized training sets. What if a GPT was fine-tuned on all of the writing and speeches of figures whom are considered the most persuasive?
Government reports continue to balloon in size, because of technological advancements (especially copy/paste).
In the age of typewriters, anything not important, or duplicate, was cut. With word processors, document sizes unnecessarily grew.
All things considered, an awkward sentence or two isn't really that much of a big deal compared to an incorrect assertion that was missed in review.
I also don't think GPT-4 is necessarily that brilliant at writing copy, but I don't really know how good/bad US government officials are so I can't really compare, but in other parts of the world I haven't really that much of an issue with it.
For the boilerplate- well sure, but I could do that with existing tools, no AI needed. Custom communications I'm sure AI can handle the very basics. but once a human replies and consideration is needed for special cases- can you imagine ChatGPT counseling you on a healthcare decision? I can't.
I can't imagine ChatGPT doing much more than being an autoattendant until a human can reply fully, except the message humans can send to the agency will be customized and drafted according to a standard intake process.
The question is, do you want an AI writing the brief and doing the research that is going to be given to an administrator who is going to decide if you are denied medical care in an emergency?
A disturbingly high proportion of US adults (at least in the US—I don't know how this looks in other countries) barely count as literate. Perhaps half[1]. Most of the rest aren't a ton better, and can't write worth a damn.
I think "on average" understates the case: at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not. Most of that remaining 10% would also see improvements from ChatGPT, except for writing they've put a great deal of effort into. It's very good at cleaning up writing, and/or tuning it to be better-suited to some purpose than the original was.
In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT and having it summarize it concisely, in very simple terms.
The world makes a lot more sense, to those who grew up taking to the written word as naturally as a duck to water, when we appreciate that the world runs on the written word, yet most people are terrible readers and worse writers.
[1] "According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level." And, nb, if you're a good reader and you're thinking "6th grade level, that's not so bad", your idea of what a "6th-grade level" looks like, is probably skewed way higher than what they mean.
Bullet list -> AI text generator -> AI text summarizer -> Bullet list
Perhaps OpenAI can optimize the process by storing the original prompt and just giving it back at the last phase.
this is quite saddening.
IIRC the high-school-graduate equivalent adult literacy rate from similar studies—as in, possessing reading skills that we hope a high school graduate would have, which basically equates to being able to read two moderate-length, moderate-complexity texts and understand them well enough to synthesize a description of their positions, and how they agree and differ—isn't much more than 20%. Four out of five people you encounter are going through life constantly struggling to understand WTF is happening or what they're supposed to do and getting by on a lot of guesswork, while all continually mis-communicating with one another. Really explains a lot.
To continue the entirety of the quote which makes it not seem quite as dammning:
"Literacy in the United States was categorized by the National Center for Education Statistics into different literacy levels, with 92% of American adults having at least "Level 1" literacy in 2019.[1] According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level.[2]
In many nations, the ability to read a simple sentence suffices as literacy, and was the previous standard for the U.S. The definition of literacy has changed greatly; the term is presently defined as the ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential.[3]"
I'd expect writing samples to get shorter, not longer, when being improved.
Since when are Government employees underpaid? The salary is typically competitive for the work being done and the benefits/pension are outrageously generous when compared to the private sector.
If you disagree, take a look at govt SWE jobs postings.
There has to be some way to explain a pay discrepancy if it exists. How does one justify receiving a lower salary for the same amount of skill and work?
I suspect (admittedly without evidence) that someone in the NSA or Marines are not primarily driven by a paycheck. Maybe that's what you mean by "true believers"?
You might need to define what you mean by "outrageously generous". I've worked in multiple public and private roles and, in general, the private ones were the ones with much more generous packages, in terms of pay and benefits.
This worldview would also mean that blue-collar employees in the government are more skilled than their private sector counterparts, no?
If that came from an organic brain we would call that clear intent.
I would however stop stick with “lying requires agency,” even if the model evaluation is crafted to be deceitful. The intent and the agency lies with the prompt creator, and therefore, IMO, the lie starts and ends there.
This is a pretty shitty assessment of places like DTIC, DoE, and NASA. Maybe there's another analysis that doesn't involve web forum commenters being the only wizards who can control the magic?
(edit) how many people do you think are aware of LLM hallucinations as opposed to inaccurate google results
LLMs will be involved in complex tool chains like everything else.
Why? Any who understands about Google or Wikipedia can understand about LLM.
If someone blindly trust and use an LLM without checking for more evidence, then they would've done the same with any other source, be it Google, the newspaper or whatever.
"In a cringe-inducing court hearing, a lawyer who relied on A.I. to craft a motion full of made-up case law said he “did not comprehend” that the chat bot could lead him astray." [1]
[1] https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-s...
Also the problem isn't necessarily about understanding the tech, but more about how it will be perceived at first. As long as every big player hypes up the tech as if it's perfect and revolutionary, I can't blame people for blindly trusting it at first(because it might as well work for most things they use it for, it will take some time before the pitfalls become apparent).
I agree there's cause for concern, and I hope it's a gradual rollout that's introspected in between each propagation of such, but I definitely think it's viable to be used no more less safely than other tooling in the government today (for better or worse).
https://digitalreadymarketing.com/wp-content/uploads/2014/08...
https://www.nytimes.com/2021/02/18/opinion/fake-news-media-a...
Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions.
Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, but not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer.
Just interesting to see.
"I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly." (completely made up and exaggerated, but I'm sure one can think of similar scenarios) [1] Good luck convincing the clerk that that's not real. You can probably get it corrected after months of your time and $$.
[1] https://www.washingtonpost.com/technology/2023/04/05/chatgpt...
[1] https://www.nytimes.com/2010/01/14/nyregion/14watchlist.html
There are 340 million people in the US. The US has one of the largest government systems that humanity is likely to ever see. How many similar cases of children being put on the terrorist watch list have there been in the past 20 years? Unless there are a lot of cases of that happening, if it's not rare (hint: it's extraordinarily rare; your example story is from 13 years ago), your premise is pure straw.
[0]: https://www.wired.com/story/face-recognition-software-led-to... [1]: https://www.theguardian.com/us-news/2023/apr/27/california-p...
>(completely made up and exaggerated, but I'm sure one can think of similar scenarios)
How about "I'm sorry sir, but the computer clearly says that you've been at the site of murder and thus killed this guy. Even though all other direct evidence contradicts this, we're gonna have to arrest you. "[1]
The best part of working on projects that used that data set in Google was learning about it from articles like this one.
[1]https://www.dailymail.co.uk/news/article-7897319/Police-arre...
We are holding it to a higher standard because this is going to be used by agencies with serious power over people's lives. It is not just to spit out, for example, a form rejection letter from a credit card.
And, nobody thinks that Microsoft wants to "screw" the government. But if something happens, the impact can be magnified significantly considering the reach and power of the state. I'm all for moving forward quickly with this tech, but there are a handful of spaces (government, medicine, aeronautics) where we need to be extremely deliberate.
This behavior is illiberal.
I feel for my kids, and all I can do is warn them...
https://theconversation.com/excel-errors-the-uk-government-h...
The vast majority of government workers are underpaid apathetic cogs in a machine who do not fit this bill.
We just saw a LAWYER use hallucinated information in his practice. A lawyer who is ostensibly rigorously trained and tested in his field.
The amount of misuse of GPT by way of sheer ignorance alone by the government will be astounding.
When talking about the pitfalls of bringing GPT-4 as a tool for government officials, the positive features are outside of the scope.
Obviously they exist, otherwise it wouldn't be proposed. But it's not our job to do a sales pitch for GPT-4.
>Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions.
No. The OP suggests that the primary users of the tool would be low-level, underpaid officials, who are mostly following directions.
Not because it's their fault as human beings - but because their positions require them to do so, and perhaps for good reasons.
In any case, direction-followers (of questionable competence and low pay grade) make up the bulk of large organizations in general as a consequence of growth. Which is why the tool is being pitched in the first place: to streamline work of people who would rather not do it in the first place.
>Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, b
As if the governments aren't prone to massive scandals even without GPT-4 in place. Try checking the news today, you may be surprised.
>not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer.
"By Microsoft" wasn't in OP's statement. Manipulating data and statistics to get the desired outcome from seemingly impartial algorithms is something our government has been engaging in for a very, very long time.
Redlining[1] and gerrymandering[2] are just two prime examples.
We don't have to make any assumptions or apply biased perspectives to say that we expect:
1)Training data to be manipulated by entities with interest in influencing the outcome;
2)Lives of countless people being adversely affected, and
3)There being a massive scandal as a result.
The government has a long track history of 1, 2, and 3 happening any time a system is introduced which lacks transparency, and large language models are the epitome of that.
The only unrealistic thing about OP's statement is that a scandal will actually happen, instead of people getting away with it.
>It's always interesting to see how people engage with new technologies on HN.
And it is even more interesting to see how people here engage with ethics.
[1]https://www.nytimes.com/2021/08/17/realestate/what-is-redlin...
[2]https://www.brennancenter.org/our-work/research-reports/gerr...
But hey, I guess all those dumb dumbs at NASA can’t tie their shoelaces.
I'm from the north-middle, sometimes I'm surprised how little people on the west coast of America think of the people on the east coast.
Above All, The NSA knows their shit. 44B+, Autonomy and Mathematicians Proves it above all, regardless of Morals.
Don't attack them on knowledge. You have to use a different approach to climb Everest.
I would only put the Mossad, KGB and <insert name of the chinese SIGINT agency> on their level.
Just communicate more clearly, you're clearly displeased and want to say something more?
2. Require human curation to procedure usable results, that will be biased because of that curation
3. Being extremely good at convincing people they should trust them ```
So it's absolutely perfect for the US government.
4. Having no legal accountability
(while arguably providing a means of avoiding or reducing accountability for those using them, "LLM says no" style.)
Every new technology initially has its own shortcut comings.
Steve jobs in his inaugural iPhone launch, used four different phones because each phone was capable of only one feature.
When was the last time you tried to make a wiki edit on a popular page?
Also scientific articles being constant don't quite matter when the "status quo" of science is asymmetrically understood.
There are doctors today still pushing the "fat is unhealthy" myth because of the sugar industry. Despite the ubiquity of modern unbiased research papers and proof.
But when you ask a doctor for advice, you're essentially getting the wetware equivalent of a stochastic parrot, because this random doctor is giving you dynamic answers coming from his/her own completely non-standardized corpus of knowledge.
> 2. Require human curation to procedure usable results, that will be biased because of that curation
> 3. Being extremely good at convincing people they should trust them
You just described politicians
It’s worse than that. Employees of the US government are definitely not underpaid, especially when you consider
What the hell was I thinking that I couldn't finish?
> 1. Lying about everything
> 2. Require human curation to procedure usable results, that will be biased because of that curation
> 3. Being extremely good at convincing people they should trust them
Finally, we automate politics!
From experience, this is sort of lazily libeling a broad cross-section of public servants. The public sector has no monopoly on useless employees, you'll find them in equal if not greater numbers in the private sector.
There are plenty of very highly skilled people in government who are quite passionate about serving the public good. The one thing you're correct on is that they're underpaid, and we're fortunate that they're passionate enough about public service and aren't focused on salary-maxing. They're stuck in a system that's managed by the whims and grandstanding of politicians, and attacked as 'the deep state' by people who want to take us back to the spoils system where every public employee is a crony.
If you think it's frustrating dealing with Google/YouTube arbitrarily and capriciously and erroneously and mercilessly shutting down your account and providing zero explanation and zero accountability and zero helpful customer service...
Now imagine that for crucial government services. If there is a flag or datapoint in your government data, then that's that. Too bad. Can't help you. That's what the computer is telling me.
This happens on a daily basis right now, for any number of human or computer errors.
Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity, and expect the GOVERNMENT to utilize it in a non-erroneous way?
This is what I'm talking about. Saying "the GOVERNMENT" is a ridiculously broad stroke, it's like saying "the INTERNET" (which, ironically, was brought to us by a particularly smart group of people working for the government).
> Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity
What's special about the tech elite that makes them any better at using these hallucinating ML models? The government includes vast numbers of specialists and researchers, and many of them are a lot smarter than we are. Government doesn't consist entirely of DMV employees.
> unskilled, underpaid employees who don’t care
One of these things is not like the others. The models may not ask for a raise but they’ll no doubt go to work finding all sorts of other excuses to raise taxes.
So humans?
Most of the "AI" fears, including the ones you picked out, are also prevalent in people. Therefore, I don't see the problem.
Humans have been inventing and using tools to increase the scale and efficiency of the means of production since time immemorial. "AI" is nothing new.
>removal of accountability?
Not holding the makers of "AI" models and users of "AI" accountable has nothing to do with "AI" itself. "AI" is just a tool like a hoe or a kitchen knife or a gun, accountability lies with whoever made and/or uses it.
1. Lying about everything
2. Require human curation to procedure usable results, that will be biased because of that curation
3. Being extremely good at convincing people they should trust them
Gee, I wonder if there’s anything that can go wrong here, when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions? And they start making decisions that affect peoples lives?
I can bet $100, that if that goes beyond any alpha test, we’ll have huge scandal in the future of how those models were purposely modified to influence government, that will make any social media scandal look like a child’s play.
EDIT: and I assume that there won’t be an information leakage - that’s the easiest problem to tackle, with separate infra that government clouds are reasonably good at.