News for hackers. Stuff that matters.
Re:New models and developer products announced at OpenAI DevDay by fancy_pantser on Monday November 06, 2023 @05:41PM (#38170170)
Open researchers are trying to shrink and speed up 138K models e.g. YaRN https://github.com/jquesnelle/yarn

It's very compelling and opens up a lot of use cases, so I've been keeping an eye out for advancements. However, inferencing on 4xA100s would be the target today for YaRN and 128K to get a reasonable token rate on their version of Mistral.

There are no comments on this comment yet.