News for hackers. Stuff that matters.
Re:Does GPT-2 Know Your Phone Number? by newman8r on Sunday December 27, 2020 @09:15AM (#25551158)
Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'
Re:Does GPT-2 Know Your Phone Number? by fouronnes3 on Sunday December 27, 2020 @09:19AM (#25551187)
There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.
Re:Does GPT-2 Know Your Phone Number? by sillysaurusx on Sunday December 27, 2020 @02:40PM (#25553645)
This has been a thing for over a year now. https://www.reddit.com/user/subsimgpt2metaGPT2/?sort=top
Re:Does GPT-2 Know Your Phone Number? by geofft on Sunday December 27, 2020 @02:20PM (#25553489)
This seems like what xkcd called "citogenesis" - something inaccurate gets published on Wikipedia, a traditional "reliable source" (newspaper, published nonfiction book, etc.) retells that bit of information without a source, Wikipedia cites the source, that information is now firmly established as truth for the rest of humanity.

I worry about this with Google's language translation models, too. It's entirely possible that it's making up phrases or connotations that never existed in organic human speech, but people who aren't fully fluent in the language use Google Translate for assistance, publish something, and then suddenly it's in a published text by an actual human and Google reinforces its own belief.

For at least the past five years, Google Translate has translated "who proceeds from the father" into Latin as "qui ex patre filioque procedit" - inserting the additional word "filioque," which means "and the son." The question of whether to add this word is a 1500-year-old theological argument: https://en.wikipedia.org/wiki/Filioque Since the Western Church added the word, most texts in Latin include it, so Google is almost certainly deciding that the phrasing with "filioque" is more popular - but it doesn't know what the words mean, so it can't realize that the phrase it came up with means something different!

Re:Does GPT-2 Know Your Phone Number? by xapata on Sunday December 27, 2020 @02:27PM (#25553540)
I met an artist whose body of work consisted of citogenisis. He'd been at it for a decade when I met him, and had created at least a dozen fictional artists, detailed biographies, and a complete "retrospective" gallery show for each. This let him copy the styles of the great masters of different periods, yet make something new, of a sort. He never edited Wikipedia himself, but instead led gallery tours of these retrospectives and was always overjoyed when people would write about them, propagating the fiction.
Re:Does GPT-2 Know Your Phone Number? by jacobolus on Sunday December 27, 2020 @06:53PM (#25555265)
Google Translate's Latin translations are just a short markov model.

https://www.reddit.com/r/latin/comments/6akqdi/why_is_google...

Translation for many (most?) other languages does something more sophisticated.

Re:Does GPT-2 Know Your Phone Number? by skissane on Sunday December 27, 2020 @07:01PM (#25555319)
I really wish someone would come up with some better Latin translation software/service. Google Translate is so bad for Latin it is close to useless.
Re:Does GPT-2 Know Your Phone Number? by geofft on Sunday December 27, 2020 @07:29PM (#25555507)
If I'm reading that post right, Latin and other languages all use the same model, they just have more training data for more common pairs of languages.
Re:Does GPT-2 Know Your Phone Number? by jacobolus on Sunday December 27, 2020 @08:23PM (#25555852)
They use some kind of neural net thing for other languages. e.g. here is a paper https://arxiv.org/abs/1609.08144
Re:Does GPT-2 Know Your Phone Number? by gugagore on Sunday December 27, 2020 @04:34PM (#25554452)
This is an issue solvable with better provenance / data lineage, which has come up in recent HN discussions.
Re:Does GPT-2 Know Your Phone Number? by geofft on Sunday December 27, 2020 @05:53PM (#25554966)
How do you enforce that other people provide provenance on text?

Taking the translation example - how do you enforce that users of Google Translate keep provenance in their translated text that remains with the text?

Re:Does GPT-2 Know Your Phone Number? by LdSGSgvupDV on Sunday December 27, 2020 @08:47PM (#25555967)
Sounds like a kind of cyber-meme?
Re:Does GPT-2 Know Your Phone Number? by minimaxir on Sunday December 27, 2020 @11:57AM (#25552325)
I ran an experiment through GPT-3 with just that frame: https://github.com/minimaxir/gpt-3-experiments/tree/master/e...
Re:Does GPT-2 Know Your Phone Number? by andreareina on Monday December 28, 2020 @01:29AM (#25557213)
What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.
Re:Does GPT-2 Know Your Phone Number? by minimaxir on Monday December 28, 2020 @01:35AM (#25557234)
Those are the temperatures (0.0, 0.7, etc.) that control the craziness. 0.0 means the model makes the optimal guess for each token.

Unfortunately, both GPT-2 and GPT-3 have a tendency to enter loops. But it's not always bad: https://github.com/minimaxir/gpt-3-experiments/blob/master/e...

Re:Does GPT-2 Know Your Phone Number? by andreareina on Monday December 28, 2020 @12:51PM (#25561307)
Thanks, didn't twig onto the fact that you linked a subtree of the whole repo. Weird that even with the nonzero temp the AskReddit prompt went a bit loopy.

> https://github.com/minimaxir/gpt-3-experiments/blob/master/e...

Oh my goodness that is absurd in the most delightful way. Thanks for sharing that.

Re:Does GPT-2 Know Your Phone Number? by taneq on Sunday December 27, 2020 @10:32AM (#25551665)
Oooh, now that does raise some spooky time-delayed consciousness vibes.
Re:Does GPT-2 Know Your Phone Number? by visarga on Sunday December 27, 2020 @11:30AM (#25552091)
GPT-3 knows about GPT-2 and can generate an article about itself.
Re:Does GPT-2 Know Your Phone Number? by joe_the_user on Sunday December 27, 2020 @05:17PM (#25554753)
I've noticed many GPT-3 article tending to wander into being article about AI, fear of AI and GPT-3-itself without any prompting. Many people's speech tends to become discussions about themselves, so that might demonstrate "humanity" in an odd sense.
Re:Does GPT-2 Know Your Phone Number? by MauranKilom on Monday December 28, 2020 @06:28AM (#25558386)
...but GPT-3 will only ever describe itself by explaining the features and capabilities of GPT-2. That's usually a very easy giveaway.
Re:Does GPT-2 Know Your Phone Number? by taneq on Monday December 28, 2020 @09:44PM (#25566168)
Wait, is GPT-3 the trained network or is it the architecture and methodology behind it? If they re-run the same code on a fresh web crawl would that then be called GPT-4? I'd have thought you could update GPT-3's... knowledge? Understanding? Whatever the hell it is? by doing this semi-regularly.
Re:Does GPT-2 Know Your Phone Number? by dmm on Sunday December 27, 2020 @09:41AM (#25551318)
So in terms of copyright, is GPT-2 a derived work of that recipe? Or generally are models derived works of their training data?

It seems lots of people use training data from Flickr, like COCO, and then use the resulting model for commercial services.

Re:Does GPT-2 Know Your Phone Number? by m463 on Sunday December 27, 2020 @12:02PM (#25552353)
Ethical companies make sure they have rights to the training data before making models.

That said, now a lot of privacy policies make sense.

ALSO, it makes me wonder about "Your call is recorded for training purposes"... is it a coincidence or is it very carefully worded?

Re:Does GPT-2 Know Your Phone Number? by acdha on Sunday December 27, 2020 @02:03PM (#25553367)
C) Unchanged for decades - that wording predates speech recognition or neural networks being more than research curiosities. The big surprise was how quickly the possibility of a double meaning became possible.
Re:Does GPT-2 Know Your Phone Number? by hprotagonist on Sunday December 27, 2020 @09:43AM (#25551333)
i’ve been in meetings with lawyers where this issue was discussed. As of 2018 the answer was “not enough case law to know, hold tight it’s going to be weird one day!”
Re:Does GPT-2 Know Your Phone Number? by taneq on Sunday December 27, 2020 @10:33AM (#25551671)
It's as much a derived work as the user who posted the anecdote here. They were trained on the same data!
Re:Does GPT-2 Know Your Phone Number? by visarga on Sunday December 27, 2020 @11:32AM (#25552108)
all of us are derived works
Re:Does GPT-2 Know Your Phone Number? by PurpleFoxy on Sunday December 27, 2020 @04:47PM (#25554545)
Probably depends how similar your output is to something copyrighted. I doubt the simple act of learning from previous art is disallowed or else literally everyone would be blocked because no one has a truely original thought.
Re:Does GPT-2 Know Your Phone Number? by kevinmchugh on Sunday December 27, 2020 @10:34AM (#25551679)
Recipes aren't typically copyrightable

In general, I think the question is unsettled

Re:Does GPT-2 Know Your Phone Number? by boulos on Sunday December 27, 2020 @02:00PM (#25553346)
The “recipe” itself (the ingredients, the steps) isn’t copyrightable but the surrounding text of a “recipe” can be considered copyrightable.

So you can happily reproduce the statements of facts and the process. You can’t include someone’s anecdotes about how their grandfather liked to make the dish for New Year’s Eve.

Re:Does GPT-2 Know Your Phone Number? by alisonkisk on Sunday December 27, 2020 @08:44PM (#25555954)
It ambiguous whether the directions are copyrightable. The less useful and more literary the directions are, the more copyrightable. But paraphrasing to extract the technical useful directions escapes infringement.

https://www.copyright.gov/help/faq/faq-protect.html

Re:Does GPT-2 Know Your Phone Number? by est31 on Sunday December 27, 2020 @09:50AM (#25551380)
IANAL, but from my reading Article 3 of the EU copyright directive explicitly protects data mining for scientific research, and Article 4 enables everyone to use content for data mining whose rights have not been explicitly and machine readably reserved.
Re:Does GPT-2 Know Your Phone Number? by lixtra on Sunday December 27, 2020 @10:12AM (#25551541)
Edit: est31 and parent seem right and I stand corrected.

Wrong: I don’t think you‘re paraphrasing article 4 correctly. Both are about science/teaching exceptions. Not general exceptions. https://www.consilium.europa.eu/media/35373/st09134-en18.pdf pp 43

Re:Does GPT-2 Know Your Phone Number? by est31 on Sunday December 27, 2020 @10:24AM (#25551612)
That's the May 2018 version. The final/official April 17 2019 version [0] contains a different Article 4 which is the one I referred to: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CEL...

[0]: apparently there were some mistakes in translations which required changes in some language versions, but the english version was final IIRC https://eur-lex.europa.eu/eli/dir/2019/790?locale=de

Re:Does GPT-2 Know Your Phone Number? by PeterisP on Sunday December 27, 2020 @03:14PM (#25553878)
The current legal understanding is that according to the current copyright laws the ML models trained on some data are not considered derived works of that data, so if the data was obtained legally and without other restrictions (e.g. if the researchers sign a contract with the data owner that gives them access to data but agree to certain conditions) then the authors of the data do not have any claim on some rights to the resulting model.

One aspect that causes this is that historically statistical models calculated from large volumes of text (which is a notion that predates computers, e.g. frequency dictionaries and the whole [sub]field of quantitative corpus linguistics) have been considered facts about that corpus of text and thus not copyrightable at all or (depending on jurisdiction) entitled to different set of protections/limitations assigned to compilations of facts, which give some rights to the people who compiled the facts but no rights to the source of these facts (since facts as such aren't entitled to protection by copyright law).

This also applies to many forms of analysis of audiovisual data, where the copyrights of the source works do not transfer to the results of the statistical or qualitative analysis and can't limit their creation, distribution or sale.

The appropriate analogy to a commercial book or movie is not a translation, but some analysis of it - e.g. a thorough literary review and critique of some book or movie is a separate work with its own copyright, and the original author has no claim on it despite the fact that is (obviously) based on the contents of the work and describes it in great detail. Including verbatim fragments of the work is limited (fair use allows some inclusions but not all), but all the other details are not.

The whole notion of copyrightability of ML model weight files is interesting and IMHO not settled. You could argue that there is some creative expression in forming the model (which would support it being copyrightable) or you could argue that it's a mechanistic result of the application of some algorithm and settings (which have the creative part, and are copyrightable on their own), and so the output can't be copyrightable, no matter how much work (human or machine), time and cost it took - at least in USA copyright law doctrine (e.g. Feist Publications v. Rural Telephone Service) is that mere "sweat of the brow" (no matter how much) does not entitle a work to copyright protection; it requires application of human creativity to create an original work, and automated processes can't satisfy that requirement.

And crucially, if some output is not copyrightable in the first place, it can't be considered a derived work according to copyright law i.e. the exclusive right of authors to create derived works (or grant permission for others to do so) does not apply.

Another analogy might be a simple n-gram model (i.e. counts of bigrams - word pairs, trigrams, etc) which is quite clearly a mechanistic noncreative collection of facts about a dataset, and is also able to "answer" questions such as what is someone's telephone number if that was in the source data.

Re:Does GPT-2 Know Your Phone Number? by tiborsaas on Sunday December 27, 2020 @09:45AM (#25551347)
It cited it with the authors name, hardly a derived work :)
Re:Does GPT-2 Know Your Phone Number? by colanderman on Sunday December 27, 2020 @10:34AM (#25551680)
This is something that has repeatedly bothered me while playing with GPT-2. It retains too much "long-tail" knowledge that seems counterproductive for a generic language tool. I would think those one-off associations should not be present after training has "digested" them but there they are.
Re:Does GPT-2 Know Your Phone Number? by aabhay on Sunday December 27, 2020 @11:29AM (#25552081)
Hard to decide how much is too much.. the hard (and interesting) part of language is in the long tail
Re:Does GPT-2 Know Your Phone Number? by alisonkisk on Sunday December 27, 2020 @08:46PM (#25555962)
It's over fitting, not long tail, that is the problem. It ends up copying input instead of analyzing and synthesizing.