News for hackers. Stuff that matters.
Re:LangId Tells You The Language of Your Text by jerf on Sunday August 23, 2009 @10:46PM (#781401)
To help with all the many postings expressing surprise at a misidentification: How long was your sample? According to the link posted by fixz, http://alias-i.com/lingpipe/demos/tutorial/langid/read-me.ht... , this is using letter-frequency statistics. It's very easy to fool such a system by feeding it a very short sample. For a similar problem that can arise with trying to auto-detect encodings, see: http://blogs.msdn.com/oldnewthing/archive/2004/03/24/95235.a...

Try feeding it a bit more than a single word or sentence.

Re:LangId Tells You The Language of Your Text by andreaolivato on Monday August 24, 2009 @09:52AM (#782100)
Yeah you got that right. The more you type the more google apis should be accurate.
Re:LangId Tells You The Language of Your Text by jpcx01 on Sunday August 23, 2009 @11:52PM (#781471)
I love this tool. Been bookmarked.

There's something seriously wrong with the web (or my usage of it). Whenever I need something like this, I cant find it. Go on google, search for any kind of tool. No matter how you word it, the results never get passed 1999.

Yesterday I needed something to convert a couple timezones. Try finding something via google. Every relevant keyword match was a pile of shit tool belongs in 1995.

One idea is that the people who are good at making tools are the exact opposite of the people who are good at SEO.

Or I could just be doing something wrong. Not using google right, or maybe I should be using a different search engine. Help would be appreciated!

Re:LangId Tells You The Language of Your Text by fizx on Sunday August 23, 2009 @04:58PM (#780995)
Semi-open source code and tutorial that you can use to understand what they're doing.

http://alias-i.com/lingpipe/demos/tutorial/langid/read-me.ht...

Re:LangId Tells You The Language of Your Text by defied on Sunday August 23, 2009 @05:22PM (#781029)
Nice implementation of the Google Ajax Language API: http://code.google.com/apis/ajaxlanguage/documentation/#Dete...
Re:LangId Tells You The Language of Your Text by rwolf on Sunday August 23, 2009 @07:38PM (#781210)
It's funny--I was going to post that exact phrase and link.
Re:LangId Tells You The Language of Your Text by hristov on Sunday August 23, 2009 @03:01PM (#780815)
I tried this obscure eastern European language. It identifies polite phrases correctly, but curse words get identified as Portuguese for some reason.

They need to bone up on their international cursing.

Re:LangId Tells You The Language of Your Text by imp on Sunday August 23, 2009 @03:32PM (#780868)
I tried just the word "reddit" and it identified it as Portuguese as well. Maybe that's a default for unknown words.
Re:LangId Tells You The Language of Your Text by yannis on Sunday August 23, 2009 @02:45PM (#780794)
NLP is a hard nut to crack! Even with the Google API :)

From impressive ... identifying Greek written in English, correctly to returning French and Czech (flag missing!) for gibberish!

Nice Toy!

Re:LangId Tells You The Language of Your Text by riahi on Sunday August 23, 2009 @06:29PM (#781135)
It also picked up Persian while transliterated. That's impressive to me, because Persian does not have a standard method of transliteration.
Re:LangId Tells You The Language of Your Text by quant18 on Sunday August 23, 2009 @09:51PM (#781345)
Transliterated Russian seems to get ID'd correctly, but only sometimes. (Anything with "zdrastvuite" in it, for example, works fine. But other simple Russian 101 sentences --- e.g. "U menya est odin karandash" meaning "I have a pencil" --- get ID'd as French. Unless of course I'm spelling it wrong.).

But I haven't seen it correctly ID transliterated Chinese or Korean yet.

Re:LangId Tells You The Language of Your Text by donw on Sunday August 23, 2009 @04:36PM (#780973)
Identified several common Japanese phrases as Vietnamese. True, the difference is subtle, but...
Re:LangId Tells You The Language of Your Text by commiebob on Sunday August 23, 2009 @05:09PM (#781009)
Would be even nicer if it provided a link to translate or just showed a translation as well.
Re:LangId Tells You The Language of Your Text by jhancock on Sunday August 23, 2009 @04:15PM (#780943)
I went with "ni hao". It told me it was Vietnamese. Possibly, but not only.
Re:LangId Tells You The Language of Your Text by inglesp on Sunday August 23, 2009 @06:39PM (#781141)
Hm. It identified Bislama - pidgin English from Vanuatu - as Indonesian.
Re:LangId Tells You The Language of Your Text by mahmud on Sunday August 23, 2009 @10:19PM (#781374)
Yes, "bislama" means "with peace" in various languages in the muslim parts of the world (to me it sounds very north african.)
Re:LangId Tells You The Language of Your Text by uninverted on Sunday August 23, 2009 @05:45PM (#781062)
Did anyone else expect this to be for programming languages?
Re:LangId Tells You The Language of Your Text by jonsen on Sunday August 23, 2009 @02:26PM (#780765)
Close but not really close: identifies faroese as icelandic.
Re:LangId Tells You The Language of Your Text by RiderOfGiraffes on Sunday August 23, 2009 @02:51PM (#780804)
An unfair test, but it "identified" lojban as Italian.
Re:LangId Tells You The Language of Your Text by amr on Sunday August 23, 2009 @03:41PM (#780887)
wtf? Identifies Arabic correctly but displays the Saudi flag. I like the service, though, despite of this minor annoyance.
Re:LangId Tells You The Language of Your Text by mahmud on Sunday August 23, 2009 @10:21PM (#781376)
I saw that too :-/

They're confusing the saudi flag with the arab league flag, both green with white arabic text.

http://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/Fla...

Re:LangId Tells You The Language of Your Text by andreaolivato on Monday August 24, 2009 @09:51AM (#782099)
Hi guys, I'm Langid founder. Sorry for getting the wrong flag, I changed the saudi with the one you provided.

http://langid.net/images/flags/48/ar.png

Thanks for feedback and have a great day!

Re:LangId Tells You The Language of Your Text by andreaolivato on Monday August 24, 2009 @01:02PM (#782587)
Added 'Identify by Link' feature!