Permutation-Invariant Neural Networks for Reinforcement Learning

Re:Permutation-Invariant Neural Networks for Reinforcement L... by bjornsing on Sunday November 28, 2021 @01:52AM (#29366775)

I haven’t read the article in depth, but transformer style self-attention is usually permutation equivariant, meaning that if you change the order of inputs then you get the same change in order of outputs, but the outputs themselves stay the same.

As I understand it all outputs are then added together / averaged into one vector. That should make the model ”hard” permutation invariant.

[ Parent | on HN ]

Re:Permutation-Invariant Neural Networks for Reinforcement L... by sdenton4 on Sunday November 28, 2021 @02:17AM (#29366869)

Most actual implementations use a position encoding, though, which breaks permutation invariance. This is a good inductive bias... word order matters, for example.

[ Parent | on HN ]

Re:Permutation-Invariant Neural Networks for Reinforcement L... by bjornsing on Sunday November 28, 2021 @02:51AM (#29367031)

Sure. But it would be very weird to use a position embedding here, when the whole point is permutation invariance.

[ Parent | on HN ]

Re:Permutation-Invariant Neural Networks for Reinforcement L... by Buttons840 on Sunday November 28, 2021 @12:21PM (#29370734)

I must not understand what transformers are. I thought they were a type of RNN where order of input matters?

Any suggestions on how to learn about transformers? Or a few sentences to get me started on the right path?

[ Parent | on HN ]

Re:Permutation-Invariant Neural Networks for Reinforcement L... by visarga on Sunday November 28, 2021 @03:38AM (#29367242)

yes, there are a bunch of permutation invariant operations - such as sum, max and pairwise interactions

[ Parent | on HN ]