News for hackers. Stuff that matters.
Re:Ask HN: Unicode text variants? by greenyoda on Wednesday March 12, 2014 @12:01AM (#7383862)
HN Search doesn't find "Unicode text variants", so it's probably not a good idea to use these characters in article titles:

https://hn.algolia.io/#!/story/forever/prefix/0/unicode%20te...

What's weird is that if I copy/paste the variant characters from the article title into HN Search, it matches a whole bunch of articles, none of which are this one.

DuckDuckGo finds this post as its first hit if you copy/paste the title into its search box, but not any other articles about "Unicode" or "variants" (just lots of random junk).

Google, on the other hand, apparently canonicalizes the text, since it returns hits on other articles about Unicode and text variants, as well as this post.

So, here we have three text search programs that behave rather differently.

Also, Firefox doesn't find any of the variant text strings on this page if you search for the normal (ASCII) characters.

Re:Ask HN: Unicode text variants? by GeneralMayhem on Wednesday March 12, 2014 @12:24AM (#7383936)
Chromium's ctrl-f also canonicalizes, at least well enough to recognize this title.
Re:Ask HN: Unicode text variants? by redox_ on Wednesday March 12, 2014 @03:38AM (#7384422)
You're right; MySQL is currently throwing an error (and therefore we don't even send the item to the Algolia engine). Gonna take a look tomorrow or the day after.

Mysql2::Error: Incorrect string value: '\xF0\x9D\x96\x86 m...' for column 'text' at row 1:

Thank you for the bug report ;)

Re:Ask HN: Unicode text variants? by xroche on Wednesday March 12, 2014 @12:31PM (#7386612)
"utf8" in mysql is limited to 16-bit range (!) ; see http://golem.ph.utexas.edu/~distler/blog/archives/002539.htm... - basically, you need to use "utf8mb4" (http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8m...)
Re:Ask HN: Unicode text variants? by redox_ on Friday March 14, 2014 @01:50PM (#7400538)
I switched our database encoding to `utf8mb4` (thank you xroche), let's understand why the text is now truncated in https://hn.algolia.io/#!/story/forever/prefix/0/unicode%20te... :)
Re:Ask HN: Unicode text variants? by arikrak on Wednesday March 12, 2014 @12:25AM (#7383937)
Bing doesn't find it when searched in plain-text, while Google can find it in both their auto-complete suggestions and in Chrome's page search. +1 Google.