News for hackers. Stuff that matters.
Re:Gemma 2: Improving Open Language Models at a Practical Si... by thomasahle on Thursday June 27, 2024 @11:25AM (#40811559)
You can just do that my shaping the attention mask, no? That also gives you an actual guarantee that no information is leaked between conversations.
Re:Gemma 2: Improving Open Language Models at a Practical Si... by suryabhupa on Thursday June 27, 2024 @12:09PM (#40812058)
In practice, and at scale, that's exactly what having <bos> and <eos> tokens allow you to easily and programmatically do.
Re:Gemma 2: Improving Open Language Models at a Practical Si... by danielmarkbruce on Thursday June 27, 2024 @04:52PM (#40814963)
You can't pack multiple examples into a single row of a matrix without knowing where one begins and one ends.