You Should Probably Pay Attention to Tokenizers

Re:You Should Probably Pay Attention to Tokenizers by p1esk on Thursday October 24, 2024 @01:31AM (#41932231)

4x seq length expansion doesn’t sound that bad.

[ Parent | on HN ]

Re:You Should Probably Pay Attention to Tokenizers by lechatonnoir on Thursday October 24, 2024 @05:03AM (#41933603)

I mean, it's not completely fatal, but it means an approximately 16x increase in runtime cost, if I'm not mistaken. That's probably not worth trying to solve letter counting in most applications.

[ Parent | on HN ]

Re:You Should Probably Pay Attention to Tokenizers by SEGyges on Friday October 25, 2024 @04:56PM (#41949679)

it is not necessarily 16x if you, e.g., decrease model width by a factor of 4 or so also, but yeah naively the RAM and FLOPs scale up by n^2

[ Parent | on HN ]