Re:Clarkesworld AI Submissions Update
by falcolas on Thursday March 23, 2023 @12:42PM
(#35277447)
Seems reasonable, especially this part:
> The fact of the matter is that we have rules. Every publication has rules. Our rules say we don’t want generated content. They also say that we don’t want horror. If someone submits horror, you wouldn’t have a problem with me completely ignoring them, so what’s the problem here?
I hope they can remain open, but I have my doubts. In the end, there's money attached to it, and if people can take a shortcut on their way to make money - even when it's against the rules - they will. To use their analogy, if there's a chance people will make $1,000 by throwing away a fork, that fork's going in the trash.
Re:Clarkesworld AI Submissions Update
by pixl97 on Thursday March 23, 2023 @01:14PM
(#35277948)
Exactly, My SMTP server doesn't want spam per my policies, but spammers don't give two shits and I have to filter absolute massive tons of spam. I had one client where their domain was like that of a popular university. For every one valid message they received, they received thousands to fake addresses that didn't exist on the server.
Re:Clarkesworld AI Submissions Update
by luke_cq on Thursday March 23, 2023 @12:42PM
(#35277464)
Really interesting, I appreciate Neil writing more about the process and philosophy than simply "we are banning AI tools."
As these models improve, I'm skeptical that we'll be able to differentiate human-written works from an edited work generated by a skilled LLM operator. It's already trivial to generate fiction content that scores as "100% human" on analysis tools.
However, sci-fi short stories are a particularly interesting case - by their very nature they are about investigating uniquely human experiences by using speculation to amplify things humans think, feel, and experience. A machine can never do that wholesale. But, what about a machine in tandem with a human that is articulating how they feel, and a premise?
Re:Clarkesworld AI Submissions Update
by neuropsha on Thursday March 23, 2023 @12:49PM
(#35277554)
And because, inevitably, there is someone in the crowd who is blinded by the tech and screaming “judge it for the quality” … No, that would be a waste of time.
This essay, which is the most in-depth analysis of this problem I’ve found so far, suggests that we’ll have to find an automated way to filter for quality, because AI-generated content is only going to get more sophisticated: https://antipodes.substack.com/p/ai-will-kill-literature-and...
The endgame seems to be the widespread deployment of AI autograders that, while they won’t be able to judge literary merit, will at least be as good as the current, mostly social, signals used today.
Re:Clarkesworld AI Submissions Update
by unkabunka on Thursday March 23, 2023 @02:32PM
(#35279257)
I am not sure sure about killing literature.
I will read the substack later as I am interested in this, but the feeling I get when I walk into a bookshop today with all the 'read me! read me!' marketing that inevitably gets pushed for whatever trendy book of the day is in whatever number 1 bestseller list or Costa Prize, along with the realisation that in 10 lifetimes I'd probably not read the books in this shop i am stood in, that this is no different to what some people think AI will do to literature.
It may make some authors not bother (or some people not bothering to become authors) but I already am overwhelmed with what I can/must/should/might read and have to deal with it now, so AI killing literature I don't think will happen.
If it makes less authors turn out the junk that fills much of the shelves I have to peruse at the moment then all the better, but I don't think I will be seeing books on shelves predominantly by AI authors.
To digress slightly, reading physical media vs online/digital does help focus one's mind in terms of taking the effort to choose what to read than the easy-grazing that can happen online. I saw a similar comment the other day to this in reference to someone getting a weekly physical paper as a good way to keep up with 'news' without being overwhelmed by online news and all the dross that brings.
Re:Clarkesworld AI Submissions Update
by ignite on Thursday March 23, 2023 @02:54PM
(#35279588)
> If it makes less authors turn out the junk that fills much of the shelves I have to peruse at the moment then all the better, but I don't think I will be seeing books on shelves predominantly by AI authors.
That sounds a lot like so many of the "this technology will never replace X", like, for example, digital cameras.
Re:Clarkesworld AI Submissions Update
by em-bee on Thursday March 23, 2023 @03:03PM
(#35279718)
digital cameras is one technology replacing another technology but not replacing the photographer or the way they take photos.
AI authors means to replace actual writers, and that's quite different.
as with the photoshop example elsewhere, it may be that more writers will use AI tools to help them, which, if done with care, may be good, that's quite different from letting an AI do all the writing. while clarkesworld currently rejects any AI involvement, someone using an AI to explore ideas or prepare an outline may be harder to detect and eventually something like that may be published, but i hope they will be successful in eliminating any AI only submissions.
Re:Clarkesworld AI Submissions Update
by Veen on Thursday March 23, 2023 @12:58PM
(#35277699)
Unfortunately, I think this is a losing battle. They can't scale the number of human readers checking for story quality, and AI can scale almost limitlessly. Automated AI detection simply doesn't work. I've experimented with all the main AI detectors, and they are woefully inaccurate, especially with GPT-4 content.
Re:Clarkesworld AI Submissions Update
by yamtaddle on Thursday March 23, 2023 @01:39PM
(#35278411)
All these publications constantly complain about being swamped with submissions, even pre-AI.
The top-tier ones should probably switch to mail-only for unknowns, with an allow-list of authors (those they've previously published, or whom they've rejected but are very interested in seeing more from) who may submit electronically. Hell, they probably should have before this.
Re:Clarkesworld AI Submissions Update
by ericbarrett on Thursday March 23, 2023 @01:53PM
(#35278643)
Riffing on "mail-only", I'd further suggest:
- Include a printout of the story in the submission packet (shipping costs)
- Also require a signed (and perhaps notarized) affidavit stating that it's entirely the author's own work*
- Require a header/submission form to be filled out by hand (this can be verified quickly with a magnifying glass)
It's not perfect; the story could still be generated by AI, for example, and autopens exist. But it would put a very manual cost and effort barrier in front of their submission process, that IMO is still not so high as to discourage legitimate entrants.
* Obviously this should be informed by legal counsel, I am not a lawyer, etc.
Re:Clarkesworld AI Submissions Update
by ghaff on Thursday March 23, 2023 @02:01PM
(#35278786)
As someone who has been on a lot of conference committees, I wouldn't necessarily require proposals to be sent physically; that would create a lot of work for us. But I do favor limiting the # per person and asking questions other than just the standard abstract that might discourage people who are just spamming everything in sight. Friction isn't always a bad thing.
Re:Clarkesworld AI Submissions Update
by ghaff on Thursday March 23, 2023 @01:28PM
(#35278198)
>Automated AI detection simply doesn't work.
When did this change? As of 2 or 3 months ago, OpenAI's tool seemed pretty reliable at separating 100% ChatGPT content from 0%. Of course, past some level of a human using a generative AI and editing it and augmenting it, it's harder. And outside of certain artificial constraints in places like schools, it's not clear that using a generative AI assistant is even "wrong." Though to the degree that floods the web, publications, conference proposals, etc. with content that is just "OK" it makes the job of people filtering it who are already flooded with mediocre but not necessarily bad content that much harder.
Re:Clarkesworld AI Submissions Update
by neuropsha on Thursday March 23, 2023 @01:16PM
(#35277995)
I don’t know why you’ve been downvoted. You are, sadly, right. Being a reader for a publication or literary agent was already a gruelling job before AI made the slush problem a hundred times worse. They don’t go into the role wanting to fire off form rejections, but there is too much hopeless perma-slush for them to do much else.
We’re going to end up with AI autograders (for quality as much as provenance) as a first pass before a human read. On the other hand, if the AIs are selective enough, they can guarantee that each passing submission can be afforded a serious, deep read… which isn’t the case right now unfortunately.
Re:Clarkesworld AI Submissions Update
by 13years on Thursday March 23, 2023 @01:59PM
(#35278746)
Unfortunately, I think this is a losing battle.
This is just a narrow representation of what is to come. Everything positioned against AI will be a losing battle. It is a skill/creativity replicator and that concept breaks the existing model of everything.
Re:Clarkesworld AI Submissions Update
by idiotsecant on Thursday March 23, 2023 @02:37PM
(#35279332)
The answer to this seems simple - require a micropayment along with the submittal. tune the amount until it's not worth spamming anymore. Return the payment when the submittal is accepted or rejected. Consider keeping the payment when the submittal is clearly AI or low quality spam.
We have all the necessary technology to do this today cleanly and efficiently. If we don't it's a social problem, not a technical one.
Re:Clarkesworld AI Submissions Update
by em-bee on Thursday March 23, 2023 @02:45PM
(#35279443)
one of the important aspects for clarkesworld is international submissions. unfortunately , while technology for micropayments exist even in eg. many african countries, i haven't yet seen any micropayment system that works globally without high transaction costs.
Re:Clarkesworld AI Submissions Update
by CuriouslyC on Thursday March 23, 2023 @01:00PM
(#35277724)
The only answer to this is to crowdsource curation. A small editorial team isn't going to be able to do anything other than cherry pick from a trusted pool.
Re:Clarkesworld AI Submissions Update
by em-bee on Thursday March 23, 2023 @02:13PM
(#35278975)
the problem with crowdsourcing is that it would probably be a pre-publication which would make it difficult to submit the story elsewhere because many magazines (including clarkesworld) do not allow that.
currently they have multiple slush readers, and depending on volume they could engage more as long as the volume does not grow indefinitely.
Re:Clarkesworld AI Submissions Update
by PuppyTailWags on Thursday March 23, 2023 @04:09PM
(#35280600)
Crowdsourcing doesn't work if you want to produce a Clarkesworld issue, because Clarkesworld has an audience expecting stories of a certain style and sense that requires training for readers.
Re:Clarkesworld AI Submissions Update
by tzs on Thursday March 23, 2023 @03:13PM
(#35279866)
With fiction though the meaning that the reader gets often is not the meaning the writer intended, and often not even related to the meaning the writer intended. The story often serves more as a prompt to get the reader to create meaning than as a way for the writer's meaning to be communicated to the reader.
It's not clear that real meaning from the writer as opposed to the appearance of meaning matters as far as how well the story works to get the reader to generate their own meaning.
Re:Clarkesworld AI Submissions Update
by hooande on Thursday March 23, 2023 @01:03PM
(#35277765)
> "As of a few minutes ago, we had received 576 submissions and processed 474 of them. 93 have resulted in bans and 7 have been marked as suspicious (a new category meaning they might have used those tools)."
I've had many stories rejected by Clarkesworld. but if I submitted something so bad they assumed GPT wrote it, I might die on the spot.
Re:Clarkesworld AI Submissions Update
by jamesgill on Thursday March 23, 2023 @01:12PM
(#35277923)
Wouldn't this be (at peast partly) resolved when it comes time to sign legal documents? At some point, the submitter has to legally vouch for their identity and authorship.
You could front-load this requirement, of course: provide proof of identity (tech already exists), then legally vouch for authorship at time of submission.
TL;DR: Raise the barrier to entry, instead of scaling the ability to review.
Re:Clarkesworld AI Submissions Update
by Jevon23 on Thursday March 23, 2023 @01:15PM
(#35277973)
A contract without teeth is worthless.
Who in the world is going to sue someone for submitting an AI-generated story to a sci-fi magazine? Who has the time, resources, and motive? You may as well just ask everyone to pinky promise that they didn’t use AI.
Re:Clarkesworld AI Submissions Update
by jamesgill on Thursday March 23, 2023 @01:18PM
(#35278024)
I agree, but I'm suggesting more than a contract; I'm also suggesting identity validation. Just the act of requiring legal proof of identity might weed out most of these attempts.
Re:Clarkesworld AI Submissions Update
by falcolas on Thursday March 23, 2023 @03:08PM
(#35279787)
Perhaps most authors wouldn't care, but identity validation would gut the concept of the pen name. And as beneficial as pen names are (allowing early women authors to publish books at all, providing an author an out if their name becomes too associated with one genre, protecting authors from fanatics), eliminating them in an attempt to prove whether a tool was used in a books creation is less than ideal.
And worse, I have my doubts that it would even work. See: counterfeiters on Amazon.
Re:Clarkesworld AI Submissions Update
by em-bee on Thursday March 23, 2023 @03:13PM
(#35279876)
identity validation would gut the concept of the pen name
as far as i understand clarkesworld submission rules, you may submit and be published under a pen name, but have to reveal your real identity to the magazine, which they will keep to themselves
Re:Clarkesworld AI Submissions Update
by dirtyid on Thursday March 23, 2023 @01:20PM
(#35278071)
Understandable but would be interesting to have a "State of AI" writing post every once in a while to see how AI measures up. Once AI starts making competent short stories, I can imagine them being good enough to write quests for next mass effect etc. So far AI content been pretty generic on that front.
Re:Clarkesworld AI Submissions Update
by nestorD on Thursday March 23, 2023 @01:49PM
(#35278575)
I think a AI-coautored section would be interesting (with an additional field to let people describe their writing process, from "I just asked for the greatest SF short story ever" to "I gave it my draft and used it as an editor").
While a lot of this writing is spam trying to make easy money, I do think that some people are doing cyborg writing in good faith and would happily report their writing as such.
Re:Clarkesworld AI Submissions Update
by ignite on Thursday March 23, 2023 @02:52PM
(#35279554)
This is similar to the older discussion about using Photoshop in photography competitions. I think that eventually moved to "everyone photoshops somewhat", which is likely where the AI tools for writing will go.
I mean, how many people use Grammarly? Doesn't that suggest changes to test based on (somewhat) ai?
Re:Clarkesworld AI Submissions Update
by galleywest200 on Thursday March 23, 2023 @07:08PM
(#35282777)
I have started giving GPT some of my own sentences I am unhappy with in prose and asking for it to repeat it in a more "active voice". I do not paste it into my writing -- I read the results and edit my original sentences as I wish, if I wish. This process will likely find me reaching out to GPT less and less as I become more familiar with the writing style I am seeking.
I think methods similar to this will become more popular with writers.
Re:Clarkesworld AI Submissions Update
by prepend on Thursday March 23, 2023 @02:09PM
(#35278912)
I think openai needs to publish an ai Levenshtein distance api. Where you feed in a block of text and it tells you how close it is to something it’s ever generated. It can keep track of everything it’s ever generated and tell us if it seems familiar or not.
> The fact of the matter is that we have rules. Every publication has rules. Our rules say we don’t want generated content. They also say that we don’t want horror. If someone submits horror, you wouldn’t have a problem with me completely ignoring them, so what’s the problem here?
I hope they can remain open, but I have my doubts. In the end, there's money attached to it, and if people can take a shortcut on their way to make money - even when it's against the rules - they will. To use their analogy, if there's a chance people will make $1,000 by throwing away a fork, that fork's going in the trash.
As these models improve, I'm skeptical that we'll be able to differentiate human-written works from an edited work generated by a skilled LLM operator. It's already trivial to generate fiction content that scores as "100% human" on analysis tools.
However, sci-fi short stories are a particularly interesting case - by their very nature they are about investigating uniquely human experiences by using speculation to amplify things humans think, feel, and experience. A machine can never do that wholesale. But, what about a machine in tandem with a human that is articulating how they feel, and a premise?
This essay, which is the most in-depth analysis of this problem I’ve found so far, suggests that we’ll have to find an automated way to filter for quality, because AI-generated content is only going to get more sophisticated: https://antipodes.substack.com/p/ai-will-kill-literature-and...
The endgame seems to be the widespread deployment of AI autograders that, while they won’t be able to judge literary merit, will at least be as good as the current, mostly social, signals used today.
I will read the substack later as I am interested in this, but the feeling I get when I walk into a bookshop today with all the 'read me! read me!' marketing that inevitably gets pushed for whatever trendy book of the day is in whatever number 1 bestseller list or Costa Prize, along with the realisation that in 10 lifetimes I'd probably not read the books in this shop i am stood in, that this is no different to what some people think AI will do to literature.
It may make some authors not bother (or some people not bothering to become authors) but I already am overwhelmed with what I can/must/should/might read and have to deal with it now, so AI killing literature I don't think will happen.
If it makes less authors turn out the junk that fills much of the shelves I have to peruse at the moment then all the better, but I don't think I will be seeing books on shelves predominantly by AI authors.
To digress slightly, reading physical media vs online/digital does help focus one's mind in terms of taking the effort to choose what to read than the easy-grazing that can happen online. I saw a similar comment the other day to this in reference to someone getting a weekly physical paper as a good way to keep up with 'news' without being overwhelmed by online news and all the dross that brings.
That sounds a lot like so many of the "this technology will never replace X", like, for example, digital cameras.
The technology frequently does.
AI authors means to replace actual writers, and that's quite different.
as with the photoshop example elsewhere, it may be that more writers will use AI tools to help them, which, if done with care, may be good, that's quite different from letting an AI do all the writing. while clarkesworld currently rejects any AI involvement, someone using an AI to explore ideas or prepare an outline may be harder to detect and eventually something like that may be published, but i hope they will be successful in eliminating any AI only submissions.
The top-tier ones should probably switch to mail-only for unknowns, with an allow-list of authors (those they've previously published, or whom they've rejected but are very interested in seeing more from) who may submit electronically. Hell, they probably should have before this.
- Include a printout of the story in the submission packet (shipping costs)
- Also require a signed (and perhaps notarized) affidavit stating that it's entirely the author's own work*
- Require a header/submission form to be filled out by hand (this can be verified quickly with a magnifying glass)
It's not perfect; the story could still be generated by AI, for example, and autopens exist. But it would put a very manual cost and effort barrier in front of their submission process, that IMO is still not so high as to discourage legitimate entrants.
* Obviously this should be informed by legal counsel, I am not a lawyer, etc.
When did this change? As of 2 or 3 months ago, OpenAI's tool seemed pretty reliable at separating 100% ChatGPT content from 0%. Of course, past some level of a human using a generative AI and editing it and augmenting it, it's harder. And outside of certain artificial constraints in places like schools, it's not clear that using a generative AI assistant is even "wrong." Though to the degree that floods the web, publications, conference proposals, etc. with content that is just "OK" it makes the job of people filtering it who are already flooded with mediocre but not necessarily bad content that much harder.
We’re going to end up with AI autograders (for quality as much as provenance) as a first pass before a human read. On the other hand, if the AIs are selective enough, they can guarantee that each passing submission can be afforded a serious, deep read… which isn’t the case right now unfortunately.
This is just a narrow representation of what is to come. Everything positioned against AI will be a losing battle. It is a skill/creativity replicator and that concept breaks the existing model of everything.
We have all the necessary technology to do this today cleanly and efficiently. If we don't it's a social problem, not a technical one.
currently they have multiple slush readers, and depending on volume they could engage more as long as the volume does not grow indefinitely.
btw, here is a thread that gives some insights how slush reading works: https://www.fantasy-writers.org/forum/slushpile-answers-clar...
I'm pleased that there is push back against this, but I don't think the people championing GPT are willing or even able to listen.
It's not clear that real meaning from the writer as opposed to the appearance of meaning matters as far as how well the story works to get the reader to generate their own meaning.
I've had many stories rejected by Clarkesworld. but if I submitted something so bad they assumed GPT wrote it, I might die on the spot.
You could front-load this requirement, of course: provide proof of identity (tech already exists), then legally vouch for authorship at time of submission.
TL;DR: Raise the barrier to entry, instead of scaling the ability to review.
Who in the world is going to sue someone for submitting an AI-generated story to a sci-fi magazine? Who has the time, resources, and motive? You may as well just ask everyone to pinky promise that they didn’t use AI.
And worse, I have my doubts that it would even work. See: counterfeiters on Amazon.
as far as i understand clarkesworld submission rules, you may submit and be published under a pen name, but have to reveal your real identity to the magazine, which they will keep to themselves
While a lot of this writing is spam trying to make easy money, I do think that some people are doing cyborg writing in good faith and would happily report their writing as such.
I mean, how many people use Grammarly? Doesn't that suggest changes to test based on (somewhat) ai?
I think methods similar to this will become more popular with writers.