Is there a reason a document could not be converted to vectors via embedding, and you’d have both?
EDIT: I see, the control vectors operate more directly upon the model, in a way embedding vectors don’t quite have access to.
EDIT: I see, the control vectors operate more directly upon the model, in a way embedding vectors don’t quite have access to.
The other explanation may be that these AI labs may be expecting more government scrutiny, and "here's a document" would probably go better than "here's some vector representation of our values" when talking to politicians.
[1] https://arxiv.org/abs/2106.09685
[2] https://vgel.me/posts/representation-engineering/
[3] https://transformer-circuits.pub/2024/scaling-monosemanticit...