Re:Text Embeddings Reveal (Almost) as Much as Text
by brendev on Friday October 13, 2023 @03:41PM
(#37874684)
Hey man this is going to sound really weird but when I just saw your name, it struck me as super familiar. Like I know I've seen your name on GitHub somewhere. I started going through your public Repos and I figured it out!
The first commit I ever made was on a fork of one of your projects over 5 years ago. I wasn't a software engineer then, I was just some guy learning to code in his free time.
Anyways I just wanted to say thank you. I know it may seem silly. But your code gave me the scaffolding I needed to take some important first steps towards where I am today. I'm now a senior software engineer/team lead!
If you were curious, here's the fork: https://github.com/brendenriggs/GottaSlackEmAll
I've got to say: I don't play any more, but I definitely miss those early day when Pokemon Go first came out. Good times.
Re:Text Embeddings Reveal (Almost) as Much as Text
by jxmorris12 on Friday October 13, 2023 @09:20PM
(#37877272)
Wow, that’s really wild! Yeah, I remember this — 8 years ago Pokémon go came out and I was working on some projects to reverse engineer the API. Now I’m reverse engineering neural networks. Go figure.
Re:Text Embeddings Reveal (Almost) as Much as Text
by josh-sematic on Friday October 13, 2023 @11:01PM
(#37877746)
These rare encounters are one of the things that make me love the internet.
Re:Text Embeddings Reveal (Almost) as Much as Text
by jgrahamc on Saturday October 14, 2023 @07:20AM
(#37879653)
I had an encounter like this in real life. Turns out Tom Scott was a really early contributor to POPFile. He was a teenager and I’d never met him. Had been enjoying his YouTube channel for a long time. One day he interviewed me at some conference and thanked me for accepting his commits when he was a teen.
Re:Text Embeddings Reveal (Almost) as Much as Text
by danielcampos93 on Friday October 13, 2023 @04:02PM
(#37874932)
Great paper! I have a question on how much this can be attributed to portions of text which GPT has already seen. Like if the same approach was followed on a new sliver of Wikipedia which GPT had not seen would the results be the same?
Re:Text Embeddings Reveal (Almost) as Much as Text
by gwern on Friday October 13, 2023 @09:37PM
(#37877385)
> A fourth option would be to simply scale the inversion model up by 10x or 100x, which would give us predictably better performance. We didn't try this because it's just not that interesting.
Scaling is always both important and interesting.
Re:Text Embeddings Reveal (Almost) as Much as Text
by jxmorris12 on Saturday October 14, 2023 @11:08AM
(#37881067)
To train a larger inversion model, I think we'll just need 16 A100s for a month or two. We can circle back in December, once the bigger model finishes training, to see that we've gotten better reconstruction performance. Fascinating!
Here are the three levels of inversion model described in the paper: 1) Training a decoder to invert embeddings gives you some similar text, but only with some overlapping words. 2) Training an encoder-decoder with the embedding projected up to a 'sequence' for the encoder works better, and gets more words back, but almost never gets the right text back exactly, especially when sequences are long. 3) Training this fancy architecture as a 'correction' model that recursively corrects and re-embeds text works really well. It turns out that search is necessary for this kind of thing, and can get many texts back exactly.
A fourth option would be to simply scale the inversion model up by 10x or 100x, which would give us predictably better performance. We didn't try this because it's just not that interesting.