News for hackers. Stuff that matters.

A comment on Byte Latent Transformer: Patches Scale Better Than Tokens.

Re:Byte Latent Transformer: Patches Scale Better Than Tokens by entilzha on Monday May 12, 2025 @06:02PM (#43967879)
Great to see our paper here again! Since the paper release, we've also released model weights here for anyone interesting in building on top of it: https://huggingface.co/facebook/blt. We also added HF Hub code to easily load the model https://github.com/facebookresearch/blt?tab=readme-ov-file#l....
[ Parent | on HN ]
Re:Byte Latent Transformer: Patches Scale Better Than Tokens by accassar on Monday May 12, 2025 @09:11PM (#43968881)
The thing that stood out for me was the use of ngram hashes as an additional feature set. My understanding of this is that its typically used as a positional feature.

Is this a limitation of the byte patches in that the positional information needs to be augmented?

[ Parent | on HN ]