News for hackers. Stuff that matters.
Re:Stealing Part of a Production Language Model by wrsh07 on Tuesday March 12, 2024 @08:05AM (#39678516)
I am curious what additional attacks knowing the last layer of an LLM enables.

Eg you go from a black box attack to some sort of white box [1]

Does it help with adversarial prompt injection? What % of the network do you need to know to identify whether an item was included in the pretraining data with k% confidence?

I assume we will see more of these and possibly complex zero days. Interesting if you can steal any non trivial % of model weights from a production model for relatively little money (compared to pretraining cost)

[1] https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm...

There are no comments on this comment yet.