OPT: Open Pre-trained Transformer Language Models

Re:OPT: Open Pre-trained Transformer Language Models by ipaddr on Tuesday May 03, 2022 @01:40AM (#31245029)

Compare three copies.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by ALittleLight on Tuesday May 03, 2022 @01:56AM (#31245140)

Or slightly randomly modify all the parameters on the copy you distribute, then it will be a match for nobody.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by ipaddr on Tuesday May 03, 2022 @12:52PM (#31250544)

You compare all three and average the variance of each value. So the more copies the better.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by bogwog on Tuesday May 03, 2022 @05:11PM (#31253644)

...or just steal it so that even if it can be traced, it's not your problem.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by vladf on Wednesday May 04, 2022 @11:39AM (#31262007)

In fairness to ipaddr, this can result in worse performance at this point.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by vladf on Wednesday May 04, 2022 @11:31AM (#31261893)

OK, this is a fun game. I think your counterattack assumes I'm picking these million weights uniformly randomly among the 175 billion. I modify my original answer: s/a million/half the weights in a deterministic subset of 2 million weights/

Select the deterministic subset by just hashing some identifier for each weight.

For any reasonable number of copies, there's a pretty unique subset between all your copies sharing a large amount of bits flipped in the same direction among this subset.

[ Parent | on HN ]