Generative Teaching Networks: Accelerating Neural Architecture Search

Re:Generative Teaching Networks: Accelerating Neural Archite... by gcucurull on Thursday February 20, 2020 @02:03PM (#22377618)

Hey, congrats on the paper, I read it a while ago and thought it was really interesting.

I tried implementing it, and the samples generated by the Teacher seem to suffer from mode collapse (as if the generator is ignoring the random vector z but not the label condition). Do you recall having that issue at some point?

I have to say I'm using a simpler generator than the one in the paper, and I'm not changing the learner architechture at each batch, only its weights.

Thanks!

[ Parent | on HN ]

Re:Generative Teaching Networks: Accelerating Neural Archite... by felipepsuch on Thursday February 20, 2020 @02:15PM (#22377710)

Thanks, I'm glad you liked it! Mode collapse was actually the one thing I never encountered during my exploration (which was the reason we looked into using GTNs as a mode-collapse solution for GANs). That said, I found meta-learning to be surprisingly hard to implement efficiently and ran into more bugs in both PyTorch and TensorFlow than I can count.

Changing the learner architecture is not that important actually so that's probably not your problem.

[ Parent | on HN ]

Re:Generative Teaching Networks: Accelerating Neural Archite... by gcucurull on Thursday February 20, 2020 @06:09PM (#22379477)

Ok, I'll keep digging to figure out where the problem might be, thanks!

[ Parent | on HN ]