News for hackers. Stuff that matters.
Re:Stealing Part of a Production Language Model by renonce on Tuesday March 12, 2024 @10:13AM (#39679760)
It’s stealing the last layer (softmax head), not an arbitrary part, also it targets “production models whose APIs expose full logprobs, or a logit bias”. Not all language model APIs have these features and this characterizes what APIs can be targeted and what can’t. These important pieces of information should have been written in the title or abstract rather than “typical API access”.
Re:Stealing Part of a Production Language Model by cs702 on Tuesday March 12, 2024 @10:59AM (#39680322)
It's still significant. When the Softmax head is the transpose of the embedding matrix, the proposed method enables extraction of the entire matrix of pretrained token embeddings from a black-box model at a shockingly low cost. If I understood this right, there's a lot of valuable information in those embeddings!
Re:Stealing Part of a Production Language Model by andy99 on Tuesday March 12, 2024 @11:08AM (#39680423)
That's not always true, many models have separate weights for the embeddings and classifier heads
Re:Stealing Part of a Production Language Model by cs702 on Tuesday March 12, 2024 @11:12AM (#39680471)
You're right. I edited my comment to qualify it with "when". Thank you!