News for hackers. Stuff that matters.
Re:Weird GPT-4 behavior for the specific string “ davidjl” by ntonozzi on Thursday June 08, 2023 @01:03PM (#36245143)
The most effective architecture for LLMs without tokenization is Meta's MEGABYTE architecture: https://arxiv.org/abs/2305.07185. The sacrifice here is that they have two different types of layers, instead of the simple uniform transformer layers.

There are no comments on this comment yet.