News for hackers. Stuff that matters.
Re:Language models cost 10x more in some languages than others by Isaiah____ on Thursday May 18, 2023 @02:41AM (#35984754)
Wonderful analysis, and the huggingface dashboard is beautiful. In the dashboard example texts pane, it would be interesting to have the tokensisation, as you did in your second graphic in the article.

There are no comments on this comment yet.