Re:OPT: Open Pre-trained Transformer Language Models
by javchz on Monday May 02, 2022 @10:35PM
(#31243783)
My bet it's probably a filter, trying to prevent create a even more realistic farmbots in social media, as they are already bad as they are now.
[ Parent
| on HN ]
Re:OPT: Open Pre-trained Transformer Language Models
by robonerd on Monday May 02, 2022 @10:39PM
(#31243820)
But they'll consider requests from government and industry.. both greater threats in the information war than any private individual.
Re:OPT: Open Pre-trained Transformer Language Models
by IAmEveryone on Tuesday May 03, 2022 @02:57AM
(#31245491)
Since “everyone” would include governments and industry as well, their restriction is guaranteed to not contain more bad actors than no restriction.
Re:OPT: Open Pre-trained Transformer Language Models
by xxpor on Monday May 02, 2022 @10:45PM
(#31243865)
Not from their perspective
Re:OPT: Open Pre-trained Transformer Language Models
by robonerd on Monday May 02, 2022 @10:47PM
(#31243885)
Of course. To somebody in Zuck's position, shoring up the power of the status quo is common sense.
Re:OPT: Open Pre-trained Transformer Language Models
by alimov on Tuesday May 03, 2022 @02:42AM
(#31245406)
I don’t know whether this is true and have no way of knowing this with any degree of certainty, but to me it seems unlikely that Mark had anything to do with this stipulation (requesting access). Although it’s not unimaginable.
Re:OPT: Open Pre-trained Transformer Language Models
by bogwog on Tuesday May 03, 2022 @12:06AM
(#31244484)
That is dumb when you consider that this thing is likely going to leak anyways. It’s inevitable, and when it does happen, it will just end up in the hands of criminals/scammers and not the general public.
Re:OPT: Open Pre-trained Transformer Language Models
by londons_explore on Tuesday May 03, 2022 @12:15AM
(#31244533)
It's super easy to watermark weights for ML models.
Just add a random 0.01 to a random weight anywhere in the network. It will have very little impact on the results, but will mean you can identify who leaked the weights.
Re:OPT: Open Pre-trained Transformer Language Models
by dotnet00 on Tuesday May 03, 2022 @01:40AM
(#31245034)
It should be easy enough to make that sort of signature very difficult to trace by simply adding a bunch of small noise to the network overall, or even simply training for a few iterations.
Re:OPT: Open Pre-trained Transformer Language Models
by zelphirkalt on Tuesday May 03, 2022 @03:13AM
(#31245593)
The person leaking it might not do so intentionally. Their computer might be compromised. Are we going to punish people for not being cybersec experts?
Re:OPT: Open Pre-trained Transformer Language Models
by ipaddr on Tuesday May 03, 2022 @12:30AM
(#31244632)
Compare two copies.
Re:OPT: Open Pre-trained Transformer Language Models
by vladf on Tuesday May 03, 2022 @01:35AM
(#31245013)
Slightly modify a million random weights by changing the least significant bit up or down.
Re:OPT: Open Pre-trained Transformer Language Models
by ipaddr on Tuesday May 03, 2022 @01:40AM
(#31245029)
Compare three copies.
Re:OPT: Open Pre-trained Transformer Language Models
by ALittleLight on Tuesday May 03, 2022 @01:56AM
(#31245140)
Or slightly randomly modify all the parameters on the copy you distribute, then it will be a match for nobody.
Re:OPT: Open Pre-trained Transformer Language Models
by ipaddr on Tuesday May 03, 2022 @12:52PM
(#31250544)
You compare all three and average the variance of each value. So the more copies the better.
Re:OPT: Open Pre-trained Transformer Language Models
by bogwog on Tuesday May 03, 2022 @05:11PM
(#31253644)
...or just steal it so that even if it can be traced, it's not your problem.
Re:OPT: Open Pre-trained Transformer Language Models
by vladf on Wednesday May 04, 2022 @11:39AM
(#31262007)
In fairness to ipaddr, this can result in worse performance at this point.
Re:OPT: Open Pre-trained Transformer Language Models
by vladf on Wednesday May 04, 2022 @11:31AM
(#31261893)
OK, this is a fun game. I think your counterattack assumes I'm picking these million weights uniformly randomly among the 175 billion. I modify my original answer: s/a million/half the weights in a deterministic subset of 2 million weights/
Select the deterministic subset by just hashing some identifier for each weight.
For any reasonable number of copies, there's a pretty unique subset between all your copies sharing a large amount of bits flipped in the same direction among this subset.