News for hackers. Stuff that matters.
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Monday May 11, 2020 @06:21PM (#23147806)
Hey! this is Joel, the maker of Artbreeder (originally called ganbreeder). The project was inspired by interactive evolutionary algorithms and novelty search (https://www.youtube.com/watch?v=dXQPL9GooyI&t=1s)

The goal was to make a tool that reflected the collaborative and explorative aspects of creativity. Also to make GAN's (and high-dimensional spaces more generally) accessible to everyone in a fun way.

Let me know if you have any questions :)

Re:Artbreeder – Extend Your Imagination with GANs by j4nt4b on Monday May 11, 2020 @07:53PM (#23148873)
Awesome work! I love this concept, but I'm a writer who's interested in collaborative generative writing. Do you know of any efforts to create something similar for writing, such as poetry or micro-fiction? If not, I'm curious if you could point me in the right direction for putting something like that together.
Re:Artbreeder – Extend Your Imagination with GANs by gwern on Monday May 11, 2020 @11:19PM (#23150270)
If you'd like to write poetry or fanfiction, I provide GPT-2-1.5b models for both on https://www.gwern.net/GPT-2 - I don't provide a slick interface for writing (although Astralite has an upcoming web interface which is very snazzy), but they could be plugged into any such interface.
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Monday May 11, 2020 @08:26PM (#23149183)
Hey, thanks! I dont know about anything collaborative and for writing. https://aidungeon.io/ Maybe collaborative to play. And then there are a lot of writers exploring fine-tuning text models like GPT2 for personal use. https://twitter.com/MagicRealismBot is one of many examples. https://towardsdatascience.com/how-to-fine-tune-gpt-2-so-you...

Maybe you should make it!

Re:Artbreeder – Extend Your Imagination with GANs by j4nt4b on Monday May 11, 2020 @09:49PM (#23149711)
Thank you!
Re:Artbreeder – Extend Your Imagination with GANs by kleer001 on Monday May 11, 2020 @08:52PM (#23149364)
If you check out the archives of NaNoGenMo over at github you'll likely find what you're looking for:

https://nanogenmo.github.io/

Check out the yearly resources.

Re:Artbreeder – Extend Your Imagination with GANs by Jack000 on Tuesday May 12, 2020 @04:42PM (#23159476)
a direct transfer of this concept to text isn't possible because the SotA for text generation is based on transformers, which do not work using latent space sampling. The directly analogous version of stylegan in the nlp world would be vq-vae, but currently it doesn't match transformers in generation quality.

There are other ways to do generative writing though, mostly with prompts.

Re:Artbreeder – Extend Your Imagination with GANs by MattRix on Tuesday May 12, 2020 @01:09AM (#23150760)
Re:Artbreeder – Extend Your Imagination with GANs by programmarchy on Monday May 11, 2020 @09:36PM (#23149629)
How did you make it so fast? Are the multiple combinations generated in parallel ahead of time once you breed images? Seems like the GPU power must be immense.
Re:Artbreeder – Extend Your Imagination with GANs by sillysaurusx on Monday May 11, 2020 @09:56PM (#23149760)
The 1024x1024 StyleGAN2 network can generate an image in around 200ms - 500ms. And Artbreeder may be powered by some beefy GPUs, so that might reduce the latency to something like 100ms. Throw in 100ms of server lag time, and it feels almost instantaneous. It's quite a technical achievement!
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Monday May 11, 2020 @10:31PM (#23149989)
Nothing fancy, just running on AWS :) Specifically, https://aws.amazon.com/ec2/instance-types/g4/ - The uploading is actually the expensive part since it can take ~2 minutes per image.
Re:Artbreeder – Extend Your Imagination with GANs by stevofolife on Monday May 11, 2020 @10:51PM (#23150096)
How much are you paying to keep this up? Is it expensive to run GPU instances like this?
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Tuesday May 12, 2020 @01:46AM (#23150899)
it's not cheap but there are ways to make it manageable, i.e a aws spot g4 instance can be as low as 15 cents an hour
Re:Artbreeder – Extend Your Imagination with GANs by vagab0nd on Saturday May 16, 2020 @01:18PM (#23205282)
How do you handle interrupt? Do you keep multiple instances in different zones?
Re:Artbreeder – Extend Your Imagination with GANs by netsec_burn on Tuesday May 12, 2020 @04:10PM (#23159075)
Joel, heads up the images that aren't supposed to be visible allow directory browsing. e.g. https://s3.amazonaws.com/artbreederpublic-shortlived/ https://s3.amazonaws.com/artbreederpublic/ https://s3.amazonaws.com/ganbreederpublic/ s3cli can navigate and download the images. Cool project!
Re:Artbreeder – Extend Your Imagination with GANs by aabbcc1241 on Tuesday May 12, 2020 @10:30AM (#23154359)
I had a similar idea inspired by JWildfire but didn't got enough momentum to make it. You made it! And you've so much content on it already. Appreciate your effort. Spent half day having fun on your website :)
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Tuesday May 12, 2020 @02:46PM (#23158003)
Thanks! Glad you liked it. JWildfire is super cool, I would love to add a fractal flame category to Artbreeder.
Re:Artbreeder – Extend Your Imagination with GANs by battery_cowboy on Tuesday May 12, 2020 @01:26AM (#23150816)
I recall you had 'a font one' in the works, but I can't find anything about it now, is that still coming? The demo is really neat in the header image!
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Tuesday May 12, 2020 @02:47PM (#23158016)
I still have the model! I started working on the image -> vector conversion (which is more complicated than I expected) and then got distracted. I should finish that up, thanks for the reminder :)
Re:Artbreeder – Extend Your Imagination with GANs by battery_cowboy on Tuesday May 12, 2020 @05:27PM (#23160106)
It's such a cool idea, imagine a webfont that changes slightly each page load.

I could make a document that slowly became more illegible after each new visitor, just for fun!

Re:Artbreeder – Extend Your Imagination with GANs by archibaldJ on Monday May 11, 2020 @08:30PM (#23149216)
Great work mate! Any plans of openning up an API for a subscription fee or something? :))
Re:Artbreeder – Extend Your Imagination with GANs by joelS on Monday May 11, 2020 @11:37PM (#23150360)
Thanks! No plans at the moment but it’s possible!
Re:Artbreeder – Extend Your Imagination with GANs by sillysaurusx on Monday May 11, 2020 @08:53PM (#23149381)
Hi Joel,

Given that nVidia has a pretty clear license saying that their model and all derivates may only be used non-commercially (https://github.com/NVlabs/stylegan2/blob/cec605e0834de5404d5...) and Artbreeder is clearly commercial (https://artbreeder.com/pricing) but is using StyleGAN (https://artbreeder.com/i?k=755521d9fe07456286a06f06), do you feel that Artbreeder is intentionally violating nVidia's license?

For what it's worth, I fully support your project and I nearly launched something similar myself. But many people choose not to do projects such as yours due to licensing considerations, and I'm wondering in general how to reconcile the moral dilemma. On one hand, nVidia almost certainly doesn't care. On the other hand, they did say very clearly "thou shalt not use this model, or any derivative thereof, commercially."

(It's tempting to believe the license only covers the code, not the model. But the same license appears in their download drive: https://drive.google.com/drive/folders/1QHc-yF5C3DChRwSdZKcx...)

Re:Artbreeder – Extend Your Imagination with GANs by joelS on Monday May 11, 2020 @10:29PM (#23149975)
Hi there! Good question, it's one I get a lot and don't mind.

First off, I am not a lawyer but have consulted them. A lot of this is a gray area and not well defined. As you mention, there is the code, the given models, models fine-tuned on those models, models trained from scratch, using styelgan reimplementations, etc. Even with the license being included there does not mean that the output of a model trained from scratch counts as derivative work. And technically I just sell things like upscaling, keeping images private, google sync etc and provide unlimited image creating for free (not sure if that would hold up in court but I think it's relevant). So, no I do not think I am violating the license. But some lawyers may feel differently.

Also, I try hard to cite everything involved. Google was thrilled by my use of the biggan model and how it showed off the model. With the amount of money I have spent on Nvidia cards, I hope they are happy too :)

and then image ownership is a whole other question. Everything in arbreeder is CC0 to keep things simple https://artbreeder.com/terms.pdf

Best, Joel

Re:Artbreeder – Extend Your Imagination with GANs by sillysaurusx on Monday May 11, 2020 @10:35PM (#23150003)
Are you saying that Artbreeder is using a model trained from scratch?

If so, was that a recent change? When I checked several months ago, your latent space sliders were compatible with nVidia's FFHQ model: https://news.ycombinator.com/item?id=23149739

The only explanation seemed to be that Artbreeder was fine-tuned from nVidia's released FFHQ model.

To rephrase the question: It seems like you were using nVidia's model (or a fine-tuned version) at some point in the past. At the time, you were also selling subscriptions via your pricing page. Do you just sort of ignore the legal implications and hope that it never becomes an issue?

(That's sometimes an effective strategy.)

Re:Artbreeder – Extend Your Imagination with GANs by mandelbrotwurst on Monday May 11, 2020 @11:11PM (#23150214)
^ IANAL, but I could imagine it being a slightly less effective strategy if he replies here in the affirmative.
Re:Artbreeder – Extend Your Imagination with GANs by detaro on Monday May 11, 2020 @09:54PM (#23149739)
How does that image link tell you which specific model they use?
Re:Artbreeder – Extend Your Imagination with GANs by sillysaurusx on Monday May 11, 2020 @10:18PM (#23149895)
Answering that requires a bit of technical detail.

If you click "Edit Genes" (e.g. on the upper-right of https://artbreeder.com/i?k=755521d9fe07456286a06f06) you'll find a bunch of sliders. StyleGAN uses sliders like this to control various facial features. For any given StyleGAN model, you can find a slider for a given attribute as follows: Create a classifier for that attribute (for example, "blond hair" probability), generate 50,000 or so outputs from the model, classify each image, then plot a fitted line through all 50k latents. You now have a blonde hair slider for your specific model.

Ok, but sliders aren't unique to nVidia's licensed model. How do we check?

Suppose Artbreeder used a model that was trained from scratch. Since it has a random initialization, its latent space wouldn't be compatible at all with nVidia's FFHQ model. Sliders on Artbreeder wouldn't work with nVidia's model, and vice-versa. A "blond hair" slider for Artbreeder would result in nonsense when applied to nVidia's model.

Artbreeder has an API, through which you can get the latent vector of any given image. You can use this to extract Artbreeder's sliders as follows: Extract the latent vector for a given Artbreeder image; move a slider to the maximum setting; extract the latent vector for the result; subtract the two latent vectors. Presto, you now have the slider.

If you add the resulting slider values to nVidia's FFHQ model, you will find that they control the same facial features on both Artbreeder and nVidia's FFHQ model. They don't match exactly – Artbreeder appears to be fine-tuned from nVidia's base model – but it's very close. So it's clearly derivative, and thus falls under nVidia's "no commercial usage for derivative work" restriction.

[dead]
Re:Artbreeder – Extend Your Imagination with GANs by sillysaurusx on Monday May 11, 2020 @09:38PM (#23149656)
I feel bad for asking the question, so maybe I deserved that.

But I asked it because I've spent many nights wondering the same thing about my work in AI, in the United States. Should I pursue something knowing it's legally questionable? How about knowing it's illegal? Big companies seem to walk the tightrope. Joel isn't a big company, yet he's guiding Artbreeder to commercial success. Should I be more ambitious?

These questions affect us all, not just Joel. So I ask it with the sole intention that it might help us navigate the strange waters we often find ourselves in, both as AI researchers and as individuals. I suspect in the coming years that we'll see many more lone-wolf type projects (https://thisfursonadoesnotexist.com/ has recently been making headlines) and legal issues are becoming more and more of an elephant in our shared room.

There are broader ethical questions at play here, too. nVidia trained their model using people's faces, without paying any of them, then claims to hold the rights to limit how people are allowed to make money on the result. Legal, perhaps, but ethical? And if it's unethical, what does that mean we should do, if anything?

Joel's project is a concrete example to point to when someone says "This project has license X, so get real and stop dreaming about starting a business." (The sentiment is shockingly common with almost any AI project, since most AI work comes from big companies with licenses like nVidia's.) Personally, I find it inspiring that someone is doing it anyway.

Re:Artbreeder – Extend Your Imagination with GANs by aspenmayer on Monday May 11, 2020 @10:34PM (#23150000)
Don’t feel bad you asked the question. I can tell from content and context that you were asking in good faith. The reply you got, well. Let’s just say it seemed in less so good faith.
Re:Artbreeder – Extend Your Imagination with GANs by battery_cowboy on Tuesday May 12, 2020 @01:24AM (#23150810)
It's sad you even had to ask, don't be sad yourself, and your contribution to the conversion taught me several important things I didn't know before about this area, so thanks a bunch!
Re:Artbreeder – Extend Your Imagination with GANs by citizenkeen on Monday May 11, 2020 @09:40PM (#23149662)
In a previous career, I was an IP lawyer in the United States, so I've honed a decent sense of what can and can't be said without running afoul of bar guidelines.

> Can we safely take your comment as legal advice in making business decisions about the use of the nVidia model?

What advice were they giving? It seemed like they were mostly asking questions. I don't think any bar in the United States would take those comments as advice, legal or otherwise.

Maybe I missed something. In the interests of quality comments and constructive feedback, maybe you could walk us through your line of reasoning?

Re:Artbreeder – Extend Your Imagination with GANs by gumby on Monday May 11, 2020 @09:37PM (#23149651)
FWIW I read sillysaurusx's comment as someone being friendly and pointing out a potential bug, not as snark or arrogance.
Re:Artbreeder – Extend Your Imagination with GANs by aspenmayer on Monday May 11, 2020 @09:32PM (#23149615)
He’s not nvidia’s lawyer either so what does it matter either way? Please don’t do this here.