News for hackers. Stuff that matters.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Supply5411 on Monday September 18, 2023 @04:54PM (#37561901)
While not exactly the same, the invention of the printing press caused a lot of controversy with the Catholic Church. With the printing press, people could mass produce and spread information relatively easily. I'm sure a lot of it was considered "low quality" (also heretical)[1]. Seems like we're going through similar growing pains now. Yes I know it's different, but it rhymes.

1. https://en.wikipedia.org/wiki/Index_Librorum_Prohibitorum

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by dougmwne on Monday September 18, 2023 @05:00PM (#37561974)
I really dislike the comparison. The printing press democratized knowledge. The LLM destroys it. LLM output is perfect white noise. Enough of it will drown out all signal. And the worst part is that it’s impossible to distinguish it from real human output.

I mean think about it. Amazon had to stop publishing BOOKS because it can no longer separate the signal from the noise. The printing press was the birth of knowledge for the people and the LLM is the death.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by woah on Monday September 18, 2023 @05:35PM (#37562400)
It used to be that a scribe would painstakingly copy a manuscript, through the process absorbing the text at a deep level. This same scribe could then apply this knowledge to his own writing, or just understand and curate existing work. The manual labor required to copy at scale employed many scribes, who formed the next generation of thinkers.

With the press, a greasy workman can churn out hundreds of copies an hour, for whichever charlatan or heretic palms him enough coin. The people are flooded with falsehoods by men whose only interest in writing is how many words they can fit on a page, and where to buy the cheapest ink.

The worst part is that it is impossible to distinguish the work of a real thinker from that of a cheap sophist, since they are all printed on the same rough paper, and serve equally well as tomorrow's kindling.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by emodendroket on Monday September 18, 2023 @05:43PM (#37562489)
Where are the good AI-generated books that serve as the positive side of this development?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by courseofaction on Monday September 18, 2023 @05:33PM (#37562379)
The LLM does democratize knowledge, but you have to be the user of the LLM, not the target of the user of the LLM.

The LLM is the most powerful knowledge tool ever to exist. It is both a librarian in your pocket. It is an expert in everything, it has read everything, and can answer your specific questions on any conceivable topic.

Yes it has no concept of human value and the current generation hallucinates and/or is often wrong, but the responsibility for the output should be the user's, not the LLM's.

Do not let these tools be owned, crushed and controlled by the same people who are driving us towards WW3 and cooking the planet for cash. This is the most powerful knowledge tool ever. Democratize it.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by halfmatthalfcat on Monday September 18, 2023 @05:39PM (#37562444)
> but the responsibility for the output is the user's, not the LLM's.

The current iteration of the internet (more specifically social media) has used the same rationality for its existence but at a level, society has proven itself too irresponsible and/or lazy to think for itself but be fed by the machine. What makes you think LLMs are going to do anything but make the situation worse? If anything, they’re going to reenforce whatever biases were baked into the training material, of which is now legally dubious.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by ForHackernews on Monday September 18, 2023 @05:35PM (#37562404)
> and can answer your specific questions on any conceivable topic

Yeah, I mean, so can I, as long as you don't care whether the answers you receive are accurate or not. The LLM is just better at pretending it knows quantum mechanics than I am.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by scarmig on Monday September 18, 2023 @05:57PM (#37562651)
Even if a human expert responds about something in their domain of expertise, you have to think critically about the answer. Something that fails 1% of the time is often more dangerous than something that fails 10% of the time.

The best way to use an LLM for learning is to ask a question, assume it's getting things wrong, and use that to probe your knowledge which you can iteratively use to prove the LLM's knowledge. Human experts don't put up with that and are a much more limited resource.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by shitloadofbooks on Monday September 18, 2023 @05:54PM (#37562625)
Asking a statistics engine for knowledge is so unfathomable to me that it makes me physically uncomfortable. Your hyperbolic and relentless praise for a stochastic parrot or a "sentence written like a choose your own adventure by an RNG" seems unbelievably misplaced.

LLMs (Current-generation and UI/UX ones at least) will tell you all sorts of incorrect "facts" just because "these words go next to each other lots" with a great amount of gusto and implied authority.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Supply5411 on Monday September 18, 2023 @06:05PM (#37562727)
My mind is blown that someone gets so little value out of an LLM. I get over software engineering stumbling blocks much faster by interrogating an LLM's knowledge about the subject. How do you explain that added value? Are you skeptical that I am actually moving and producing things faster?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by lxgr on Monday September 18, 2023 @06:09PM (#37562774)
My mind is also blown by how much people seemingly get out of them.

Maybe they’re just orders of magnitude more useful at the beginning of a career, when it’s more important to digest and distill readily-available information than to come up with original solutions to edge cases or solve gnarly puzzles?

Maybe I also simply don’t write enough code anymore :)

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Supply5411 on Monday September 18, 2023 @06:16PM (#37562863)
I'm very far from the beginning of my career, but maybe I see a point in your comment, because I frequently try technologies that I am not an expert in.

Just yesterday, I asked if Typescript has the concept of a "late" type, similar to Dart, because I didn't want to annotate a type with "| null" when I knew it would be bound before it was used. Searching for info would have taken me much longer than asking the LLM, and the LLM was able to frame the answer from a Dart perspective.

I would say that that information neither "important to digest" nor "readily available."

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by lxgr on Monday September 18, 2023 @07:16PM (#37563440)
Ah yes, gathering information in a particular unfamiliar area probably describes it better.

For me, it's been able to give very good answers when they were within the first few Google results when searched for using the proper terms (but the value is in giving you these terms in the first place!).

For questions from my field, it's been wildly hallucinating and producing half-truths, outdated information, or complete nonsense. Which is also fair, because the documentation where the answers could be found is often proprietary, and even then it's either outdated or outright wrong half of the time :)

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by pests on Monday September 18, 2023 @06:06PM (#37562751)
I agree with you but at what point does it change? Aren’t we all just stochastic parrots? How do we ourselves choose the next word in a sentence?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by skydhash on Monday September 18, 2023 @06:38PM (#37563096)
We use languages to express ideas. Sentences are always subordinate to the ideas. It's very obvious when you try to communicate in another language you're not fluent in. You have the thought, but you can't find the words. The same thing happens when writing code, taking ideas from the business domain and translating it into code.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by __loam on Monday September 18, 2023 @06:54PM (#37563249)
God dammit please stop comparing these things to brains. Stop it. It's not even close.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by barrysteve on Monday September 18, 2023 @06:58PM (#37563277)
If you wish to make an apple pie, first you must make the universe from scratch. (carl sagan)

We can generate thoughts that are spatially coherent, time aware, validated for correctness and a whole bunch of other qualities that LLMs cannot do.

Why would LLMs be the model for human thought, when it does not come close to the thoughts humans can do every minute of every day?

Aren't we all just stochastic parrots, is the kind of question that requires answering an awful lot about the universe before you get to an answer.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by lxgr on Monday September 18, 2023 @07:24PM (#37563540)
In my view, one big learning from LLMs is that yes, more often than not we are just stochastic parrots. And more often than not that's enough!

But sometimes we're more than that: Some types of deep understanding aren't verbal or language-based, and I suspect that these are the ones that LLMs will have the hardest time getting good at. That's not to say that no AI will get there at all, but I think it'll need something fundamentally different from LLMs.

For what it's worth, I've personally changed my mind here: I used to think that the level of language proficiency that LLMs demonstrate easily would only be possible using an AGI. Apparently that's not the case.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by ogogmad on Monday September 18, 2023 @06:08PM (#37562771)
I've used gpt4 to decompile pdf back to latex, got it to write new browser plugins nearly autonomously, and had it do complicated things based only on a vague description of what I wanted. Your comment betrays a shallow dogmatism, and lack of curiosity, which I find truly shocking in a website about tech.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by __loam on Monday September 18, 2023 @06:52PM (#37563228)
This happened to me looking up am obscure c library. It just confidently made up a function that didn't actually exist in the library. It got me unstuck but you can really fuck yourself if you trust it blindly.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by lxgr on Monday September 18, 2023 @06:05PM (#37562736)
For a librarian, they’re confidently asserting factual statements suspiciously often, and refer me to primary literature shockingly rarely.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by arrowsmith on Monday September 18, 2023 @06:46PM (#37563170)
In other words they behave like a human?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @05:22PM (#37562254)
> the worst part is that it’s impossible to distinguish it from real human output

Doesn't that make human content look bad in the first place?

If we can't distinguish a Python book written by a human engineer or by ChatGPT, how can we demonstrate objectively that the machine-generated one is so much worse?

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Nashooo on Monday September 18, 2023 @05:47PM (#37562531)
The problem is not that no one can distinguish it. It's that the intended audience (beginners in Python in your example) can't distinguish it and are not able to easily find and learn from trusted sources.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by emporas on Monday September 18, 2023 @06:11PM (#37562803)
If beginners in Python programming are not capable of visiting python.org, assuming they are genuinely interested in learning Python, it would be very questionable how good their knowledge on the subject can really be.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:18PM (#37562878)
100% agreed.

I've seen many developers using technologies without reading the official documentation. It's insane. They make mistakes and always blame the tech. It's ludicrous...

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:01PM (#37562689)
Aren't there already bad Python books written by humans?

I bet ChatGPT can come up with above-average content to teach Python.

We should teach beginners how to prompt engineer in the context of tech learning. I bet it's going to yield better results than gate-keeping book publishing.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by nneonneo on Monday September 18, 2023 @06:07PM (#37562757)
There are, but it used to take actual time and effort to produce a book (good or bad), meaning that the small pool of experts in the world could help distinguish good from bad.

Now that it’s possible to produce mediocrity at scale, that process breaks down. How is a beginner supposed to know whether the tutorial they’re reading is a legitimate tutorial that uses best practices, or an AI-generated tutorial that mashes together various bits of advice from whatever’s on the internet?

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:16PM (#37562855)
Personally I don't subscribe to the "best practices" expression. It implies an absolute best choice, which, in my experience, is rarely sensible in tech.

There are almost always trade-offs and choosing one option usually involves non-tech aspects as well.

Online tutorials freely available very rarely follow, let's say, "good practices".

They usually omit the most instructive parts, either because they're wrapped in a contrived example or simplify for accessibility purposes.

I don't think AI-generated tutorials will be particularly worse at this to be honest...

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:04PM (#37562725)
Another great contribution would be fine-tuning open source LLMs on less popular tech. I've seen ChatGPT struggling with htmx, for example (I presume the training dataset was small?), whereas it performs really well teaching React (huge training set, I presume)
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by mostlylurks on Monday September 18, 2023 @06:18PM (#37562884)
That argument might work for content which serves a purely informational purpose, such as books teaching the basics of programming languages, for instance, but it doesn't work for art (e.g. works of fiction) because most of the potential for a non-superficial reading of a work relies on being able to trust that there is an author that has made a conscious effort to convey something through that work, and that that something can be a non-obvious perspective on the world that differs from that of the reader. AI-generated content does not have any such intent behind it, and thus you are effectively limited to a superficial reading, or if were to instist on assigning such intent to AI, then at most you would have one "author" per AI model, which additionally has no interesting perspectives to offer, simply those perspectives deemed acceptible in the culture of whatever group of people developed the model, no perspective that could truly surprise or offend the reader with something they had not yet considered and force them to re-evaluate their world view, just a bland average of their dataset with some fine tuning for PR etc. reasons.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by jameshart on Monday September 18, 2023 @10:59PM (#37565145)
We can distinguish it. That's what publishers and editors do. It's also what book buyers for book chains used to do. Reviewers, writing for reputable publications, with their own editors and publishers, as well.

Humans, examining things, and putting a reputation that matters on the line to vouch for it.

The fact that Amazon doesn't want to have smart, contextually aware humans look at and evaluate everything people propose to offer up for sale on their storefront doesn't mean it can't be done. Same as how Google doesn't want to look at every piece of content uploaded to YouTube to figure out if it's suitable for kids, or includes harmful information. That's expensive, so they choose not to do it.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by vosper on Monday September 18, 2023 @05:03PM (#37562024)
> The printing press democratized knowledge

That's true, but it also allowed protestant "heretics" to propagate an idea that caused a permanent schism with the Catholic church, which led to centuries of wars that killed who-knows-how-many people, up to recent times with Northern Ireland.

(Or something like that, my history's fuzzy, but I think that's generally right?)

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by omeid2 on Monday September 18, 2023 @05:17PM (#37562186)
While AI may generate new ideas, that is a mere accident of generating "content". :)
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by bbarnett on Monday September 18, 2023 @05:49PM (#37562560)
I thought it was a king wanting a divorce, and as he couldn't get it from the catholic church, created his own.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by mmcdermott on Monday September 18, 2023 @06:02PM (#37562698)
Henry VIII created the Church of England in 1534 for the purposes of granting himself an annulment. Most histories count Martin Luther's 95 Theses as beginning of the Reformation in 1517 (a crisp date for a less-than-crisp event; Luther did not originally see himself as protesting the Roman Catholic Church). The Protestant Reformation was a heterogeneous movement from the beginning.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by TRiG_Ireland on Monday September 18, 2023 @06:02PM (#37562709)
Not really, no. It was Luther who kick-started Protestantism. Henry VIII attempted to supplant the Pope, and kind of slid into Protestantism by accident.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by vladms on Monday September 18, 2023 @06:05PM (#37562733)
That was the case just for the anglican church, which is only one "part" of the reformation.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by verve_rat on Monday September 18, 2023 @06:12PM (#37562810)
Protestantism started in Germany with Martin Luther nailing his theses to a church door. Henry's reproductive problems came later and where only sort of related.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by gnarbarian on Tuesday September 19, 2023 @06:32PM (#37577911)
Long before that in 1054, was the East - West Schism that split the Catholic and Orthodox churches. https://en.wikipedia.org/wiki/East%E2%80%93West_Schism
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rockemsockem on Monday September 18, 2023 @05:17PM (#37562181)
I really feel like you can't have used any advanced LLMs if you legitimately think the out put "perfect white noise". The results that you can get from an LLM like GPT-4 are incredibly useful and are providing an enormous amount of value to lots of people. It isn't just for generating phony information to spread or having it do your work for you.

I get the most value out of asking for examples of things or asking for basic explanations or intuitions about things. And I get so much value from this that I really think the printing press is the most apt comparison.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by softg on Monday September 18, 2023 @05:25PM (#37562293)
The problem is advanced LLMs are controlled by large corporations. Powerful local models exist (in part thanks to Meta's generosity oddly enough) and they're close to GPT-3.5, but GPT-4 is far ahead of them and by the time other models reach to that point whatever OpenAI or Antropic, Meta etc. have developed behind closed doors could be significantly better. In that case open models will be restricted to niche uses and most people will use the latest model from a giant corp.

So it is possible that LLMs will centralize the production and dissemination of knowledge, which is the opposite of what people think the printing press did. I hope I'm wrong and open models can challenge/overtake state of the art models developed by tech giants, that would be amazing.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by courseofaction on Monday September 18, 2023 @05:37PM (#37562428)
Precisely. I spent weeks learning about cybersecurity when GPT-4 first came out, as I could finally ask as many stupid questions as I liked, get detailed examples and use-cases for different attacks and defenses, and generally actually learn how the internet around me works.

Now it refuses, because OpenAI's morals apparently don't include spreading openly available knowledge about how to defend yourself.

Scary. I have also been using it to generate useful political critiques (given a particular theoretical tradition, some style notes, and specific articles to critique, it's actually excitingly good). What if OpenAI decides that's a threat? What reason do we have to think that a powerful institution would not take this course of action, in the cold light of history?

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by blibble on Monday September 18, 2023 @05:50PM (#37562563)
how do you know what you learnt wasn't completely made up gibberish?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Supply5411 on Monday September 18, 2023 @06:39PM (#37563115)
The same way you know that the things you learn from a person isn't made up gibberish: You see how well it explains a scenario, how well it lines up with your knowledge and experience, and you sample parts to verify.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by blibble on Monday September 18, 2023 @07:20PM (#37563486)
how do you know they didn't learn from the garbage generator too?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by SargeZT on Monday September 18, 2023 @10:00PM (#37564821)
You are literally describing the fundamental problem of truth in philosophy and acting as if it's different because a computer is involved at one step in the chain.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by courseofaction on Tuesday September 19, 2023 @05:15AM (#37567343)
Because it works.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by mostlylurks on Monday September 18, 2023 @06:30PM (#37563000)
What you say is not in conflict with AI-generated content being white noise. Even if you find some piece of AI-generated content useful, it is still white noise if it is merely combining pieces of information found in its dataset and the result is posted online or published elsewhere. There is no signal being added in that process, and it pollutes the space of content. Humans are also prone to doing this, but with the help of AI, it becomes a much larger issue.

"Signal" would mean new data, which is by definition not possible via LLMs trained on publicly available content, since that means the data is already out there, or new and meaningful ideas or innovations beyond just combining existing material. I have not seen LLMs accomplish the latter. I consider it at least possible that they are capable of such a feat, but even then the relevant question would be how often they produce such things compared to just rearranging existing content. Is the proportion high enough that unleashing floods of AI-generated content everywhere would not lower the signal-to-noise ratio from the pre-AI situation?

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Supply5411 on Monday September 18, 2023 @05:12PM (#37562132)
You could argue that speech is literally noise that drowns out the signals of your environment. If you just babbled, it would be useless, but instead you use it intelligently to communicate ideas. LLM output is a new palette with which humans can compose new signals. We just have to use it intelligently.

Prompt engineering is an example of this. A clever prompt by a domain expert can prime an LLM interaction to yield better information to the recipient in a way that the recipient themselves could not have produced on their own.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by jedberg on Monday September 18, 2023 @05:10PM (#37562118)
If you asked the Church back then, they would tell you that the printing press was the death of truth, because to them only the word of god was truth, and only the church could produce it.

It's all just a matter of perspective.

Yes, right now it looks like white noise, just like back then it looked like white noise which could drown out the religious texts. But we managed to get past it then and I'm sure we'll manage now.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by duskwuff on Monday September 18, 2023 @05:20PM (#37562223)
This is an astoundingly bad take. Surely you aren't trying to suggest that original, factual, human-authored content has no more inherent value than randomly generated nonsense?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @05:25PM (#37562291)
The discussion here is that we're not able to distinguish them.

If we cannot distinguish, I'd argue they have similar value.

They must have. Otherwise, how can we demonstrate objectively the higher value in the human output?

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by SketchySeaBeast on Monday September 18, 2023 @05:32PM (#37562377)
If they were of similar value would there be a problem with the deluge?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:28PM (#37562977)
Can't the deluge be delusional or an overreaction at best?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by emodendroket on Monday September 18, 2023 @05:43PM (#37562500)
I can't distinguish between pills that contain the medicine that I was prescribed and those than contain something else entirely. Therefore taking either should be just as good.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @05:57PM (#37562662)
Really. Are you comparing a complex chemical analysis required to attest the contents of a pill to reading text?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by emodendroket on Monday September 18, 2023 @06:20PM (#37562903)
It depends, is the text of a technical nature? How exactly is one to know they're being deceived if, to take one of the examples that has been linked in this discussion, they receive a mushroom foraging guide but the information is actually AI-generated?
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:25PM (#37562952)
You first check who published it. Is the author an expert in the matter with years, perhaps decades in the industry?

Heck, we always did that since before GPT.

Good authors will continue to publish good content because they have a reputation to protect. They might use ChatGPT to increase productivity, but will surely and carefully review it before signing off.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by emodendroket on Monday September 18, 2023 @07:54PM (#37563874)
"We" certainly did not "always" do that before.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Tuesday September 19, 2023 @02:51AM (#37566455)
Really? You buy books without searching anything about who wrote it?

If yes, well, there's the problem then. It's not AI, but the lack of guidance and research skills in support of the process of choosing a book.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by emodendroket on Tuesday September 19, 2023 @10:16AM (#37570279)
Is it about "me" specifically? Anyway, how do I know the biography of the author I find isn't also AI-generated at this rate? Or that the purported author actually wrote the book? Your solution still ultimately depends on there being non-generated information somewhere down the line.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Tuesday September 19, 2023 @06:50PM (#37578138)
I thought it would be pretty obvious that one should look for biographical data from external, independent sources. If the person has earned a reputation in any industry, they'll probably have articles on respectable publishers, would have presented in conferences, maybe even have patents, etc. Just Google their name. Then Google what's associated with them. If one doesn't find anything solid, discard the book. It takes no more than 5-10 minutes to recognize a solid reputation like that.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by snailmailman on Monday September 18, 2023 @06:04PM (#37562724)
They can be distinguished. They are just becoming more difficult to. Its slightly-more difficult, but also the amount of garbage is overwhelming. AI can spit out entire books in moments that would take an individual months or years to write.

There are lots of fake recipe books on amazon for instance. But how can you really be sure without trying the recipes? It might look like a recipe at first glance, but if its telling you to use the right ingredients in a subtly-wrong way, its hard to tell at first glance that you won't actually end up with edible food. Some examples are easy to point at, like the case of the recipe book that lists Zelda food items as ingredients, but they aren't always that obvious.

I saw someone giving programming advice on discord a few weeks ago. Advice that was blatantly copy/pasted from chat GPT in response to a very specific technical question. It looked like an answer at first glance, but the file type of the config file chat GPT provided wasn't correct, and on top of that it was just making up config options in attempt to solve the problem. I told the user this, they deleted their response and admitted it was from chatGPT. However, the user asking the question didn't know the intricacies of "what config options are available" and "what file types are valid configuration files". This could have wasted so much of their time, dealing with further errors about invalid config files, or options that did not exist.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:10PM (#37562800)
They can indeed distinguish them, I agree. So why the fuss?

I think the concern is that bad authors would game the reviews and lure audiences into bad books.

But aren't they already able to do so? Is it sustainable long term? If you spit out programming books with code that doesn't even run, people will post bad reviews, ask for refunds. These authors will burn their names.

It's not sustainable.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by failuser on Monday September 18, 2023 @06:36PM (#37563083)
What’s the fuss about spam? You can distinguish it from useful mail? What’s the fuss about traffic jams? You’ll get there eventually.

The LLM allow DDoS attack by increasing the threshold needed to check the books for gibberish.

It’s not like this stream of low quality did not exist before, but the topic is hot and many grifters try LLMs to get a quick buck at the same time.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by snailmailman on Monday September 18, 2023 @06:28PM (#37562982)
It doesn't need to be sustainable as one author or one book. These aren't real authors. Its people using AI to make a quick buck. By the time the fraud is found out, they've already made a profit.

They make up an authors name. Publish a bunch of books on a subject. Publish a bunch of fake reviews. Dominate the search results for a specific popular search. They get people to buy their book.

Its not even book specific, its been happening with actual products all over amazon for years. People make up a company, sell cheap garbage, and make a profit. But with books, they can now make the cheap garbage look slightly convincing. And the cheap garbage is so cheap to produce in mass amounts that nobody can really sort through and easily figure out "which of these 10k books published today are real and which are made up by ai".

It takes time and money to produce cheap products at a factory. But once these scammers have the AI generation setup, they can just publish books on loop until someone ends up buying one. They might get found out eventually, and they will have to pretend to be a different author, and they just repeat the process.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by geraldwhen on Monday September 18, 2023 @06:53PM (#37563236)
It’s sustainable if you can automate the creation of amazon seller accounts. Based on the number of fraudulent Chinese seller accounts, I’d say it’s very likely automated or otherwise near 0 cost.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by duskwuff on Monday September 18, 2023 @07:27PM (#37563577)
> Some examples are easy to point at, like the case of the recipe book that lists Zelda food items as ingredients

As an aside, the case you're thinking of was a novel, not a recipe book. Still embarrassing, but at least it was just a bit of set dressing, not instructions to the reader.

https://www.cnet.com/culture/zelda-breath-of-the-wild-recipe...

> I saw someone giving programming advice on discord a few weeks ago. Advice that was blatantly copy/pasted from chat GPT in response to a very specific technical question.

This, on the other hand, is a very real and a very serious problem. I've also seen users try to get ChatGPT to teach them a new programming language or environment (e.g. learning to use a game development framework) and ending up with some seriously incorrect ideas. Several patterns of failure I've seen are:

1) As you describe, language models will frequently hallucinate features. In some cases, they'll even fabricate excuses for why those features fail to work, or will apologize when called out on their error, then make up a different nonexistent feature.

2) Language models often confuse syntax or features from different programming languages, libraries, or paradigms. One example I've heard of recently is language models trying to use features from the C++ standard library or Boost when writing code targeted at Unreal Engine; this doesn't work, as UE has its own standard library.

3) The language model's body of "knowledge" tends to fall off outside of functionality commonly covered in tutorials. Writing a "hello world" program is no problem; proposing a design for (or, worse, an addition to) a large application is hopeless.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by SargeZT on Monday September 18, 2023 @10:08PM (#37564857)
> The language model's body of "knowledge" tends to fall off outside of functionality commonly covered in tutorials. Writing a "hello world" program is no problem; proposing a design for (or, worse, an addition to) a large application is hopeless.

Hard disagree. I've used GPT-4 to write full optimizers from papers that were published long after the cutoff date that use concepts that simply didn't exist in the training corpus. Trivial modifications were done after to help with memory usage and whatnot, but more often than not if I provide it the appropriate text from a paper it'll spit something out that more or less works. I have enough knowledge in the field to verify the corectness.

Most recently I used GPT-4 to implement the paper Bayesian Flow Networks, a completely new concept that I recall from the comment section on HN people said "this is way too complicated for people who don't intimately know the field" to make any use of.

I don't mind it when people don't find use with LLMs for their particular problems, but I simply don't run into the vast majority of uselessness that people find, and it really makes me wonder how people are prompting to manage to find such difficulty with them.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by skydhash on Monday September 18, 2023 @06:10PM (#37562789)
I'm not a native English speaker, but ChatGPT answers in each interaction I had with it sound bland. And I dislike the bite-sized format of it. I'm reading "Amusing Ourselves to Death" by Neil Postman and while you may agree or disagree with his take, he developed it in a very coherent way, exploring several aspects. ChatGPT's output falls into the same uncanny valley as the robotic voice from text to speech software, understandable, but no human does write that way.

ChatGPT as an autocompletion tool is fine, IMO. As well as generating alternative sentences. But anything longer than a paragraph falls back to the uncanny valley.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:21PM (#37562919)
I totally agree. So why are people so worried about books being written by ChatGPT?

These pseudo-authors will get bad reviews, will lose money in refunds, burn their names.

It's not sustainable. Some will try, for sure, but they won't last long.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Dylan16807 on Tuesday September 19, 2023 @01:37AM (#37565949)
There's too many names and it's too cheap to do this.

The equilibrium shifts to making it much harder to find good books, and that was already hard enough.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by failuser on Monday September 18, 2023 @06:34PM (#37563060)
If you ask LLM something you know you can distinguish noise from good output. If you ask LLM something you don’t know then how do you know if the output is correct? There are cases where checking is easier than producing the result, e.g. when you ask for a reference.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rmbyrro on Monday September 18, 2023 @06:39PM (#37563108)
Book buyers should give themselves primarily by who's the author, I think.

Choose a book from someone that has a hard earned reputation to protect.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by failuser on Monday September 18, 2023 @07:33PM (#37563660)
There is bootstrapping process of learning which authors in that field have good reputation before you know anything about the field. That is being disrupted by LLMs as well, though.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by mostlylurks on Monday September 18, 2023 @06:39PM (#37563117)
A piece of human-written content and a piece of AI-written content may have similar value if we cannot distinguish between them. But if you can add the information that the human-written content was written by a human to the comparison, the human-written content becomes significantly more valuable, because it allows for a much deeper reading of the text, since the reader can trust that there has been an actual intent to convey some specific set of ideas through the text. This allows the reader to take a leap of faith and put in the work required to examine the author's point of view, knowing that it is based on the desires and hopes of an actual living person with a lifetime of experience behind them instead of being essentially random noise in the distribution.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by jedberg on Monday September 18, 2023 @05:24PM (#37562274)
No not at all, I'm not sure why you would even think that.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by duskwuff on Monday September 18, 2023 @05:29PM (#37562346)
As I read it, your parent comment suggests that the distinction in quality and utility between human-authored and AI-generated content is merely "a matter of perspective", i.e. that there is no real distinction, and that they're both equally valuable.

If you actually meant something else, you should probably clarify.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by bcjordan on Monday September 18, 2023 @05:39PM (#37562443)
To clarify, are you claiming all LLM produced output is "Randomly generated nonsense"?

It seems there is disagreement on this point among participants in this discussion, and that may be what is causing confusion.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by dambi0 on Monday September 18, 2023 @06:17PM (#37562870)
Suggesting clarification to suit your imaginary inferences seems puzzling. The parent post pointed out that perspectives on authorship have a historical precedent, I didn’t see the value judgement your reading suggested.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by lovemenot on Monday September 18, 2023 @06:24PM (#37562945)
I am not the person to whom you replied. I understood their comment to be about paradigms shifting through social awareness of the limits and opportunities of new technology.

It can be both true that right now predominantly low quality content emanates from LLMs and at some future time the highest quality material will come from those sources. Or perhaps even right now (the future is already here, just unevenly distributed).

If that was their reasoning, I tend agree. The equivalent of the Catholic Church in this metaphor is the presumption human-generated content's inherent superiority.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by __loam on Monday September 18, 2023 @07:02PM (#37563301)
LLMs are inherently approximations of collective knowledge. They will never be better than their training sets. It's a statistical impossibility.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by rileyphone on Monday September 18, 2023 @06:28PM (#37562979)
That's Wittgenstein's argument.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by SketchySeaBeast on Monday September 18, 2023 @05:17PM (#37562193)
I'd argue that giving a group with unique thoughts and ideas a voice is different than creating a noise machine.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by jedberg on Monday September 18, 2023 @05:19PM (#37562216)
I think the jury is still out on whether an LLM produces ideas any more or less unique than most humans. :)
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by tzs on Monday September 18, 2023 @09:39PM (#37564690)
The printing press made books cheap relative to hand copied books, but they were still expensive for most people.

Before the printing press two books cost around the same as a 2 story cottage.

Afterwards a couple books would be about a month of wages for a skilled worker.

That greatly limits ones ability to drown out anything with books.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by OfSanguineFire on Monday September 18, 2023 @05:07PM (#37562063)
> The printing press democratized knowledge.

Not for centuries. Due to the expense of the technology and the requirement in some locations for a royal patent to print books, the printing press just opened up knowledge a bit more from the Church and aristocracy to the bourgeoisie, but it did little for the masses until as late as the 1800s.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by ls612 on Monday September 18, 2023 @05:17PM (#37562184)
A big part of this is that literacy didn’t come to the masses until the 1800s. But in England and the Netherlands you had (somewhat) free press by the late 1600s and early 1700s.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by pacman2 on Monday September 18, 2023 @05:22PM (#37562247)
I was told publishers dont promote a good book anymore these days. They ask how many instagram followers do you have?

Maybe the self-publishing and BoD will decline in the long term due to ML white noise and publishers are a sign of quality again.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by omeid2 on Monday September 18, 2023 @05:16PM (#37562177)
It is fascinating that we can (finally?) make a "reasonable" distinction between things (human and AI generated content) that we find almost impossible to distinguish from each other.

And yet, all along "A distinction without difference" was(?) a solid criticism.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by peab on Monday September 18, 2023 @05:05PM (#37562037)
I'm reminded of the Library of Babel
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by jakderrida on Monday September 18, 2023 @05:18PM (#37562197)
>The printing press democratized knowledge. The LLM destroys it.

Damn, what a comeback. Don't hold back any punches now.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by ben_w on Monday September 18, 2023 @06:17PM (#37562876)
> LLM output is perfect white noise.

Not even close to white noise. White noise, in the context of the token space, looks like this:

auceverts exceptionthreat."<ablytypedicensYYY DominicGT portaelight\- titular Sebast Yellowstone.currentThreadrition-zoneocalyptic

which is literally the result of "I downloaded the list of tokens and asked ChatGPT to make a python script to concatenate 20 random ones".

No, the biggest problem with LLMs is that the best of them are simultaneously better than untrained humans and yet also nowhere near as good as trained humans — someone, don't remember who, described them as "mansplaining as a service", which I like, especially as it (sometimes) reminds me to be humble when expressing an opinion outside my domain of expertise, as it knows more than I do about everything I'm not already an expert at.

Specific example: I'm currently trying to use ChatGPT-3.5 to help me understand group theory, because the brilliant.org lessons on that are insufficient; unfortunately, while it knows infinitely more than I do about the subject, it is still so bad it might as well be guessing the multiple choice answers (if I let it, which I don't because that would be missing the point of using a MOOC like brilliant.org in the first place).

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by mrighele on Monday September 18, 2023 @06:29PM (#37562995)
> Amazon had to stop publishing BOOKS because it can no longer separate the signal from the noise.

That's because they are trying very hard not to check what they are selling, hoping that their own users and a few ML algorithms can separate the signal from the noise for them. It seems to me that the approach is no longer working, and they should start doing it by themselves.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by __loam on Monday September 18, 2023 @06:47PM (#37563186)
People comparing the AI bullshit spigot to the printing press are clowns.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by BarryMilo on Monday September 18, 2023 @05:37PM (#37562419)
You're implying that what is being produced has actual value, the problem is they're acting in patently bad faith. Weep not for the spammers.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by gamepsys on Monday September 18, 2023 @05:53PM (#37562607)
The rhyme has a lot to do with how existing power structures handle a sudden increase in the amount of written text generated. In this comparison, they both try to apply the breaks. Banned books didn't work well for the Catholic Church. I think increasing QA for Amazon might actually help their book business. Of course, a book seller has a greater responsibility to society than to make money.
Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Barrin92 on Monday September 18, 2023 @06:26PM (#37562959)
>similar growing pains

For what it's worth, these 'growing pains' took the form of the wars of religion in Europe, which in Germany killed up to 30% of the population, that's in relative terms significantly worse than the casualties of World War I and II. So maybe the Catholic Church had a point

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by lovemenot on Monday September 18, 2023 @06:41PM (#37563131)
>> So maybe the Catholic Church had a point

Is that really the take-away? If the Catholic Church had not been so belligerent, those wars would not have been needed. Now that we are past that time, we should surely thank those combatants who helped disseminate knowledge against the Church whose interest was in hoarding it.

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by Barrin92 on Monday September 18, 2023 @07:08PM (#37563352)
I think that's a pretty bad reading of history frankly. The Church didn't hoard knowledge, in fact they were arguably the primary preservers of knowledge and disseminator of it, through the monastic tradition in Medieval Europe. Many thousands of which were destroyed during the religious wars, which is a common theme as far sectarian wars go. They are first and foremost destroyers of knowledge.

More importantly I certainly wouldn't want to live through that period for any reason, and much less repeat it. If an ordinary printing press caused that much chaos I'm not sure I want to figure out what one on steroids is going to do

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by thaumasiotes on Monday September 18, 2023 @09:42PM (#37564707)
> the invention of the printing press caused a lot of controversy with the Catholic Church

> https://en.wikipedia.org/wiki/Index_Librorum_Prohibitorum

The example is from the sixteenth century, but the printing press is from the seventh century.

I don't think the Catholic Church bothered to take any notice at all?

Re:Citing “rapid evolution of generative AI,” Amazon limits ... by langsoul-com on Monday September 18, 2023 @11:04PM (#37565180)
The incentive system is completely different. The new AI generated content is for a quick buck, just spamming out content because $1 x 10,000 is a lot.

If it was written with the aid of AI, that's different. At least someone tried to make something good and just used avalible tools to enhance the quality.