News for hackers. Stuff that matters.
Re:Robots are writing poetry, and many people can’t tell the... by gwern on Saturday May 07, 2022 @07:22PM (#31299614)
FWIW, the best source of GPT-3 poetry, if you want to see what it can do beyond the cursory discussion in OP, remains my own https://www.gwern.net/GPT-3#poetry page. (Pointing out, among other things, that GPT-3 can't rhyme and so Andrew Brown's sample is a bit misleading about capabilities.) I'm surprised it's neither mentioned nor linked, since there is not that much GPT-3 poetry out there...

OP argues that GPT-3's poems 'fall dead' and never become 'a memorable unity'. I will just say that for me, a number of GPT-3's poems like "Uber-Poem" (https://www.gwern.net/GPT-3#uber-poem) or "The Universe Is A Glitch" (https://www.gwern.net/GPT-3#the-universe-is-a-glitch) struck me as a good deal more lively and memorable than the vast majority of the poems I puzzle through reading the New Yorker or Paris Review.

Re:Robots are writing poetry, and many people can’t tell the... by wolverine876 on Saturday May 07, 2022 @08:25PM (#31300031)
> a number of GPT-3's poems like "Uber-Poem" (https://www.gwern.net/GPT-3#uber-poem) or "The Universe Is A Glitch" (https://www.gwern.net/GPT-3#the-universe-is-a-glitch) struck me as a good deal more lively and memorable than the vast majority of the poems I puzzle through reading the New Yorker or Paris Review.

I read them and they are impressive and memorable - because they were generated by a computer. If a person wrote them they would be very unremarkable; nothing about them would stand out, in insight or language or form. What is remarkable is that a computer generated text that is passable as human poetry. If a high schooler or non-artistic college student wrote them, I wouldn't be surprised.

(Also, and maybe I'm too literal here, if poems are seen as 'puzzles', that will miss the point for 99% of them.)

Re:Robots are writing poetry, and many people can’t tell the... by behringer on Sunday May 08, 2022 @02:01AM (#31301721)
You're right. The first poem is complete nonsense as expected. The second poem first half could be absolutely fascinating, but completely devolved into nonsense too.

Perhaps using this gpt3 stuff for poetry could start you off like a writing prompt.

Re:Robots are writing poetry, and many people can’t tell the... by ummwhat on Sunday May 08, 2022 @06:07AM (#31302684)
You're biased by a priori knowing you were about to read computer generated text. I hereby invoke the right to a fair and public Turing test.
Re:Robots are writing poetry, and many people can’t tell the... by rolandog on Sunday May 08, 2022 @08:11AM (#31303336)
Does the Turing test contemplate that the person doing the evaluation should not be wary they are performing one?

I would falsely classify some of my fast-typing family members and friends as bots due to the high amount of grammar mistakes or failure to respond in full to a set of questions.

Re:Robots are writing poetry, and many people can’t tell the... by gwern on Sunday May 08, 2022 @09:18PM (#31309518)
I was using puzzle in a double-sense: that they were puzzles rather than art, and that I was puzzled that with an entire world of poetry to choose from they chose these, in which I could find such little merit.

As for highschooler/college poetry, you must've read some pretty amazing selections if you'd rate those as the norm.

Re:Robots are writing poetry, and many people can’t tell the... by pmoriarty on Saturday May 07, 2022 @09:38PM (#31300437)
"if poems are seen as 'puzzles', that will miss the point for 99% of them"

What is the point of 99% of them?

Re:Robots are writing poetry, and many people can’t tell the... by wolverine876 on Saturday May 07, 2022 @11:11PM (#31300904)
> What is the point of 99% of them?

It depends on the poem. It's not a tool (and of course even tools have many uses, including uses for which they weren't intended) or a project, with some defined purpose.

Many, encountering poems, understandably don't know what to do next. It's outside their experience. It's not a puzzle or a secret code; much of the poetry of the past was the popular art of its day, before recorded and broadcast arts. Everyone read it, if they could read. If John Lennon had been an artist in the 19th century, and could only play music for the people in a small room and wanted to convey their vision much ore widely, wouldn't poetry be a way to convey it? Publish a book and reach the world. What would Lennon write if restricted only to words? Imagine what Lennon would put into each syllable, trying to say it all without a note of an instrument, without even the intonation of a voice.

So read it. Don't be fooled by its length - it's nothing at all like other text with the same number of words - but be patient, give it time as if it was a short story. Be curious, explore it, explore the mind behind it and its vision, completely independent of yours (put your own aside completely - you are interested in them for this moment). I've read much poetry and it often is obscure to start - in fact, the first reading usually is just basic familiarization - and it reveals itself more and more as I explore it. It can be a transportive experience.

Re:Robots are writing poetry, and many people can’t tell the... by normac2 on Sunday May 08, 2022 @12:21AM (#31301265)
Just FYI, John Lennon actually was a popular poet! [1] Mostly nonsense verse, influenced by Lewis Carroll.

[1] https://en.wikipedia.org/wiki/In_His_Own_Write

Re:Robots are writing poetry, and many people can’t tell the... by bryanrasmussen on Sunday May 08, 2022 @01:24AM (#31301568)
Yeah it was popular, because it was Lennon, but it wasn't particularly good in its own right.
Re:Robots are writing poetry, and many people can’t tell the... by brnaftr361 on Saturday May 07, 2022 @09:45PM (#31300459)
I like Tolstoy's conclusion in What is Art?, which I'm doing an injustice by summarizing as "universal emotional communication" which is contrasted with paid work, technical prowess, esoteric work, formulaic-derivative or pure aesthetics - anything that requires effort and "cultivation" to understand is not art.
Re:Robots are writing poetry, and many people can’t tell the... by wolverine876 on Saturday May 07, 2022 @10:49PM (#31300797)
> anything that requires effort and "cultivation" to understand is not art.

Tolstoy said that? Where? His work require a bit of effort and cultivation ... so does an incredible amount of art. Emotional communication does not at all exclude effort or cultivation; in fact, you can 'hear' a lot more if you sharpen your perception.

Cultivation seems hard to distinguish from any learning. Who is uncultivated, a four year old? Is art only what a four year old understands? That greatly limits what it can communicate to people who have become cultivated beyond that.

Re:Robots are writing poetry, and many people can’t tell the... by brnaftr361 on Sunday May 08, 2022 @03:11AM (#31301954)
My interpretation of what he intended more closely aligns with the word "pretentious". Artistic contributions that are veiled behind complexity and technique, or require explanation and close examination, and exposure. I can't speak to the native Russian texts, but the English translations of his work, I would surmise, could be understood by anyone. In the preface to the copy of What is Art? that I read, it was reported that Tolstoy struggled with the topic for some time, and that he struggled to adhere to the principals he laid that define art. But again, I'm butchering it, and suggest you read it for yourself.

The peasant work song is art, Wagner's leitmotifs are pretentious.

Re:Robots are writing poetry, and many people can’t tell the... by wolverine876 on Monday May 09, 2022 @04:12AM (#31311732)
Thanks for sharing that about Tolstoy.

> The peasant work song is art, Wagner's leitmotifs are pretentious.

So you have declared!

Re:Robots are writing poetry, and many people can’t tell the... by nescioquid on Saturday May 07, 2022 @10:29PM (#31300684)
I've never seen Tolstoy's attempt at a such definition before, but from what you describe, it reminds me of what Joyce had Stephen Daedalus think about art (I wonder if it was informed at all by Tolstoy).

As I recall it, the idea is that art starts with an unmitigated cry which is then made abstract, universal, in some way. I see no way for this to occur without some technical prowess, however.

One can surely be a noble and delight in the way in which the peasants and serfs conduct their culture. But to surmise that their art was artless (i.e. without τέχνη) seems foolish and makes me think there must be more to it than what you've suggested in your comment. Otherwise, were I to throw a fit in the street, that would qualify as art; but were I to do it in hexameter, that would not be art.

I think you are in essence right about what you're arguing, however. No cry comes from a machine, after all.

Re:Robots are writing poetry, and many people can’t tell the... by brnaftr361 on Sunday May 08, 2022 @03:23AM (#31301999)
It's not so much that art had ought to be without prowess, but that the predominating reason for art to exist is as a medium of communication of a message, of emotion, to everyone. Art for art's sake, art designed only to convey the technical abilities of an artist isn't art because it isn't communicating the frame of reference of an artist, it's showing their ability to cleanly transition from dark to light values. It isn't intended to convey the mournful sadness and doesn't impute the viewer with the sense of voyeurism a la Repin's Ivan the Terrible and His Son Ivan on 16 November 1581 but as I qualified in the parent, I'm doing the text an injustice, it's a great work.

https://en.m.wikipedia.org/wiki/Ivan_the_Terrible_and_His_So...

Re:Robots are writing poetry, and many people can’t tell the... by jameshart on Saturday May 07, 2022 @10:35PM (#31300720)
One definition is that a poem is a mechanism for communicating a feeling from inside one human's mind into another's.

In which case no AI-generated poem (prior to actual GAI) can properly be called a poem, since it isn't generated out of a feeling.

On the other hand:

If a human, who is feeling something, but is unable to summon the words to express that feeling, prompts an ML program to help them generate some words that they think communicate that feeling - then shares that with an audience and successfully communicate that way...

who's to say that's not poetry?

Re:Robots are writing poetry, and many people can’t tell the... by state_less on Saturday May 07, 2022 @11:07PM (#31300889)
I was hanging out with a girl who worked to verify the authenticity of social media profiles. She saw a number of computer generated profiles. One that stuck with her had a line, “Every year I look for a raise, but it never comes.”

Since it’s computer generated, it’s not a story about someone’s experience. It’s not a communique from one human to another. I thought it was like tap dancing on the keyboard of the human mind, and when the computer lands an evocative line, which we thought this was, it has something to say about ourselves.

My hope with computer generated poetry is that we might be able to use it know ourselves better.

Re:Robots are writing poetry, and many people can’t tell the... by jameshart on Sunday May 08, 2022 @12:51PM (#31305733)
That’s a very good point. If a computer program can generate words that make you feel something, does it even matter that there was no intent behind the program that selected those words?

A landscape can be breathtakingly beautiful. It doesn’t matter that it wasn’t created specifically to invoke that feeling - just that the forces of nature conspired to create something so exquisite is enough.

Maybe the same goes for computerized creativity. Pieces of beauty found among the landscape of possible things it can emit are like natural wonders, rather than works of art.

Re:Robots are writing poetry, and many people can’t tell the... by jumbododo on Sunday May 08, 2022 @05:51AM (#31302609)
> I thought it was like tap dancing on the keyboard of the human mind, and when the computer lands an evocative line, which we thought this was, it has something to say about ourselves.

To me this is no different conceptually as 'seeing' a face in the grill of a car or the front of a building. A sort of mental version of 'Pareidolia'.

Re:Robots are writing poetry, and many people can’t tell the... by 52-6F-62 on Sunday May 08, 2022 @12:21AM (#31301266)
Hardened materialists?
Re:Robots are writing poetry, and many people can’t tell the... by drusepth on Sunday May 08, 2022 @03:33AM (#31302040)
The "American poet Andrew Brown" here -- I'm honored, I don't think anyone has ever called me a real poet before!

Looks like the full poem they're referring to is:

> I think I’ll start to rain,

> Because I don’t think I can stand the pain,

> Of seeing you two,

> Fighting like you do.

For the record, the original tweet [1] doesn't mention rhymes, but the replies [2] do: I say generations that actually rhyme are "pretty rare" and that only half of poems generated with the prompt I was sharing were even worth editing into something good.

Of course, that part doesn't get quoted in articles like this...

[1] https://twitter.com/drusepth/status/1293937354978271234

[2] https://twitter.com/drusepth/status/1294024126814752768

Re:Robots are writing poetry, and many people can’t tell the... by sebmellen on Saturday May 07, 2022 @08:27PM (#31300045)
The video for The Universe is a Glitch is amazing: https://youtube.com/watch?v=5wl29laSOU4
Re:Robots are writing poetry, and many people can’t tell the... by practice9 on Saturday May 07, 2022 @08:55PM (#31300231)
Do you know if there is a page where you can guess whether a given poem is GPT-n generated or written by a human?

Would be a cool small experiment to check if people can tell the difference, with actual data

Re:Robots are writing poetry, and many people can’t tell the... by gwern on Sunday May 08, 2022 @09:31PM (#31309594)
There is not that I know of, although you could probably cobble one together from my samples.
Re:Robots are writing poetry, and many people can’t tell the... by nullc on Saturday May 07, 2022 @08:17PM (#31299941)
Have you heard of anyone attempting a transformer better specialized for poetry? ISTM that GPT3's tokenization isn't ideal as makes it needlessly complex to model how words sound.

Maybe reducing the vocabulary to simple english so every word is a single token and using some embedding that was trained using word sounds, or something along those lines?

Re:Robots are writing poetry, and many people can’t tell the... by gwern on Sunday May 08, 2022 @09:23PM (#31309549)
I talked with one or two people were interested in further poetry work but I haven't heard of anything since. The most interesting recent news is someone claiming that InstructGPT can be prompted to do what looks like genuine rhyming, but they didn't share their prompt and I've been too busy to followup.

The vocab tokenizing already mostly works, as you can see from the results. I think the solution really is as simple as switching to a byte encoding like ByT5, and perhaps someone spending 30 minutes to run a few megabytes of text through an IPA phonetic encoder to enrich the training corpus. Rhyming is just not that hard or complex compared to the things neural nets already do. It's purely artifactual. Then one can focus on more interesting and challenging tasks, like curating high-quality corpuses for finetuning.

Re:Robots are writing poetry, and many people can’t tell the... by nullc on Tuesday May 10, 2022 @06:19PM (#31333415)
Yeah, the challenge there is both understanding the word sounds and enough of the world to say something clever.

It shouldn't just help with rhyming, puns that depend on understanding how words sound. One of these models could easily be vastly super human at cross-language puns if they only understood the sound of words. :P

Re:Robots are writing poetry, and many people can’t tell the... by wizwit999 on Saturday May 07, 2022 @07:43PM (#31299731)
Honestly English poetry is just very bad.
Re:Robots are writing poetry, and many people can’t tell the... by zdragnar on Saturday May 07, 2022 @09:19PM (#31300364)
I agree. I think a lot of this falls to how poetry is taught in schools.

There's a bunch of emphasis on the basic mechanics, but there is almost no insight at all into what makes a poem compelling.

Take, for instance, the humble haiku. I was taught it was three lines with a set number of syllables. No insight into there Japanese language, the meaning behind recurring themes, or even it's origins in collaborative poetry called renku.

You can draw a line from haiku, to wood block prints, to dada and surrealism, to beatnicks and the Beatles.

Western / English poetry is treated even worse. Lessons usually boil down to "write out words so that some rhyme in the right places".

There are some fantastic English poets throughout history and today, buried in a sea of unfortunate imitations.