Re:tiktoken: OpenAI’s Tokenizer
by voz_ on Friday December 16, 2022 @02:02AM
(#34011549)
I do not like the name.
[ Parent
| on HN ]
The comparison is with an alternative called "huggingface". At least "Tiktoken" gives some clue as to what the component is for.
Re:tiktoken: OpenAI’s Tokenizer
by jeroenvlek on Friday December 16, 2022 @03:19AM
(#34012148)
Huggingface is a lot more than just a tokenizer.
Re:tiktoken: OpenAI’s Tokenizer
by charcircuit on Friday December 16, 2022 @03:18AM
(#34012137)
This project has nothing to do with TikTok though
And the name implies there’s a connection. Which is problematic.
Of course, it's because you are not tokenizing it right. It's tik-token.
(Could we say it is kind of a garden-path word?)
Re:tiktoken: OpenAI’s Tokenizer
by _Algernon_ on Friday December 16, 2022 @06:01AM
(#34013128)
It really doesn't. Does every name that has "book" or "face" in the name, imply a connection to Facebook?
Maybe not but Facebooker, Facebooked, Facebocker, and Facebooken all sound like they do.
Re:tiktoken: OpenAI’s Tokenizer
by _Algernon_ on Friday December 16, 2022 @06:33AM
(#34013415)
In none of your examples the resulting words ("booked", "bocker", "booken") are common nouns which also happen to relate directly to the product. It is different for a tokeniser to include "token" in its name.
Which is clearly why they pointed out the use of the whole name of the service in both cases.
Re:tiktoken: OpenAI’s Tokenizer
by _Algernon_ on Friday December 16, 2022 @06:57AM
(#34013589)
I'm aware they both include the full name. That was not the point I was making. Since the difference I was highlighting was unclear:
facebook-ed: Includes the whole name, but nothing else. Obviously bad and not OK.
tik-token: Includes the whole name. Also includes a generic common noun that relates to the product at hand, and is completely fine to include in the name. Appart from that "tik" can not be said to "imply a connection to tiktok". This is fine.
Re:tiktoken: OpenAI’s Tokenizer
by TechBro8615 on Friday December 16, 2022 @02:55PM
(#34019406)
I see "tiktok en" like en.wikipedia.org or microsoft.com/en
When I first encountered huggingface (without having seen its logo), I had to think of these guys: https://en.wikipedia.org/wiki/Alien_(creature_in_Alien_franc... - so, by trying to pick a domain that sounded welcoming (and was still available), they achieved the exact opposite (at least in my case)...
Re:tiktoken: OpenAI’s Tokenizer
by thatguyagain on Friday December 16, 2022 @04:03AM
(#34012421)
It's just a little humor