Re:Gemma 2: Improving Open Language Models at a Practical Si...
by thomasahle on Thursday June 27, 2024 @11:25AM
(#40811559)
You can just do that my shaping the attention mask, no?
That also gives you an actual guarantee that no information is leaked between conversations.
Re:Gemma 2: Improving Open Language Models at a Practical Si...
by suryabhupa on Thursday June 27, 2024 @12:09PM
(#40812058)
In practice, and at scale, that's exactly what having <bos> and <eos> tokens allow you to easily and programmatically do.
Re:Gemma 2: Improving Open Language Models at a Practical Si...
by danielmarkbruce on Thursday June 27, 2024 @04:52PM
(#40814963)
You can't pack multiple examples into a single row of a matrix without knowing where one begins and one ends.
To compensate for that, you can pack multiple examples in the same sequence. This is there EOS and BOS come in, as they indicate to the model that the two parts of the sequence are not related.