News for hackers. Stuff that matters.
Re:Open Questions about Generative Adversarial Networks by tasdfqwer0897 on Tuesday April 09, 2019 @10:18PM (#19621202)
Hey, I wrote this! Happy to answer questions.
Re:Open Questions about Generative Adversarial Networks by formalsystem on Wednesday April 10, 2019 @01:15AM (#19622053)
So let's say I'm in a regime where I'd like to train something like a language model RNN on a private text dataset. If I train the RNN directly on the data then I'm essentially leaking private data to the output. Do you know of any approaches to generate fake data using GANs, train a language model on that fake data and still get a good classifier that does well on test data while still quantifying how much private data is being leaked?

Related: do GANs work well as a data augmentation technique and can their exact contribution to how much a model could potentially be improved be quantified.

EDIT: Added some clarifications

Re:Open Questions about Generative Adversarial Networks by yorwba on Wednesday April 10, 2019 @04:50AM (#19622920)
You could train your classifier on a labeled private dataset, use it to classify unlabeled samples from a public dataset, then throw away your original classifier and use its classifications on the public dataset to train a replacement. That new classifier will only leak information that could also have been derived from the public dataset, if it had been labeled.

I can think of two ways that procedure can fail:

1. If some aspects of the private data aren't represented in the public dataset (e.g. codewords in classified documents), the final classifier won't recognize them. On the other hand, I'm not sure whether there's an actual use case for making a classifier public that's required to work on data that can only be obtained from a non-public source.

2. If some aspects of your labels are sensitive (e.g. which political opinions you agree with) then labeling the public dataset retains that information. In that case, you'd likely need to apply techniques similar to those studied for removing racial or gender bias from a model.

Re:Open Questions about Generative Adversarial Networks by p1esk on Wednesday April 10, 2019 @01:57AM (#19622234)
You can train rnn on encrypted data
Re:Open Questions about Generative Adversarial Networks by formalsystem on Wednesday April 10, 2019 @02:01AM (#19622255)
Sure but the output would still leak data. E.g: You give the RNN the phrase "Chase Bank" and it outputs "Sell the stock".

Encrypting the data secures the data loading part but it's not like your model didn't learn anything.

Re:Open Questions about Generative Adversarial Networks by p1esk on Wednesday April 10, 2019 @02:40AM (#19622394)
The output would be encrypted of course. You’d decrypt it on your end. Whoever hosts the model can’t know what it learned, without breaking your encryption.
Re:Open Questions about Generative Adversarial Networks by yorwba on Wednesday April 10, 2019 @03:21AM (#19622557)
Unless you use fully homomorphic encryption (way too expensive for machine learning), the model can't learn anything without breaking your encryption. So you fixed the leak only at the cost of making the model completely useless.
Re:Open Questions about Generative Adversarial Networks by p1esk on Wednesday April 10, 2019 @09:25AM (#19624382)
Source? Unless the encryption actually destroys information I don’t see why it would necessarily make the job harder for the rnn.
Re:Open Questions about Generative Adversarial Networks by yorwba on Wednesday April 10, 2019 @11:16AM (#19625524)
If you want to learn the function y = f(x), but present x and y as encrypted data x' and y', the problem becomes finding g such that g(x') = y' = encrypt(y) = encrypt(f(x)) = encrypt(f(decrypt(x'))), so the model doesn't just have to learn f, it also has to solve for the decryption and encryption routines.
Re:Open Questions about Generative Adversarial Networks by p1esk on Wednesday April 10, 2019 @02:38PM (#19627549)
You look at this as a human, but for an rnn this would be the same as learning to map x to y where both x and y are in a different language. It wouldn't need to also learn how to translate that language into ours.

There might be encryption schemes which make learning such a "language" harder for an rnn, but it does not necessarily mean these methods are harder to break. So it might be possible to find/design an encryption method which is easy to learn (map x to y), but hard to break. I don't know much about encryption though, so I might be wrong.

Re:Open Questions about Generative Adversarial Networks by yorwba on Wednesday April 10, 2019 @05:33PM (#19629017)
If an encryption scheme is easy to learn in the sense that if f is learnable, then so is g, then it is also easy to break.

Proof: You can easily train an RNN to return the n-th letter of its input and ignore everything else. Take that function to be f_n. Then g_n is the function that takes the encrypted input and returns the n-th letter, encrypted. If the encryption scheme is learnable in the above sense, then you can also train an RNN to perform that task. You can use multiple of those RNNs to extract an encryption of each individual letter in the input, which is equivalent to having the input encrypted with a substitution cipher. Substitution ciphers are so easy to break with frequency analysis that people have been doing it by hand for more than a thousand years.

Conversely, if you have a secure encryption scheme, then all statistical regularities in the input should be obfuscated so that they cannot be exploited by an RNN.

Re:Open Questions about Generative Adversarial Networks by p1esk on Wednesday April 10, 2019 @07:50PM (#19629803)
Good point. I didn't realize the goal of a good cipher is to remove statistical regularities (i.e. make it indistinguishable from random noise).
Re:Open Questions about Generative Adversarial Networks by woliveirajr on Tuesday April 09, 2019 @11:01PM (#19621431)
I'm interested in using GAN as a method to make authorship attribution more robust.

Using complex NN models (let's say, LSTM) doesn't give much better results than simple neurons with deep layers. Seems that the model extraction (how to represent some author style) is the only point that matters, and it then easily forged (i.e., not hard to reproduce the desired style on purpose).

I'm trying to use GAN as a mean of desconstruction, of removal of the ease patterns to see if some NN can use subtle characteristics to identify the author.

Re:Open Questions about Generative Adversarial Networks by tasdfqwer0897 on Tuesday April 09, 2019 @11:05PM (#19621439)
So you are worried that your existing attribution method is too focused on 'obvious' attributes and you want to see if you can make it focus on less obvious things?

IIUC, that's something that's been looked at in the ML Fairness literature. See this paper for example: http://www.aies-conference.com/wp-content/papers/main/AIES_2...

Re:Open Questions about Generative Adversarial Networks by clickok on Tuesday April 09, 2019 @10:50PM (#19621372)
You mention possible score functions in Problem 5, so I have a query about that. This has been bugging me for a bit-- is there a characterization of applicable loss functions for GANs? I'm curious about their effects on the results, but also if there's room for different losses. Or do we capture most desirable behavior using the listed score functions?
Re:Open Questions about Generative Adversarial Networks by tasdfqwer0897 on Tuesday April 09, 2019 @10:58PM (#19621411)
Hmm, I'm not sure what you mean by applicable loss functions?

I'll answer what I think you're asking and you can tell me if I got it wrong:

There's been a lot of effort spent on coming up with different loss functions for GANs, but https://arxiv.org/abs/1711.10337 shows that, according to the metrics in problem 5, they don't really improve results compared to the original GAN loss function.

There's something called a Wasserstein GAN https://arxiv.org/abs/1701.07875 that you may have heard about, but IMO the useful thing to take away from that paper is their 'gradient penalty' technique and not the new loss function.

Re:Open Questions about Generative Adversarial Networks by RSchaeffer on Wednesday April 10, 2019 @10:39AM (#19625115)
Why is a VAE not a generative model?
Re:Open Questions about Generative Adversarial Networks by govg on Wednesday April 10, 2019 @11:27AM (#19625630)
They are generative models.