Show HN: Franc – Detect natural languages

Re:Show HN: Franc – Detect natural languages by michaelmior on Friday October 03, 2014 @08:25AM (#8405169)

It would be interesting to see comparisons with language detection libraries written in other languages as well. Not just in terms of runtime, but also accuracy. Actually, it seems like this would be useful as a separate project to help the decision-making process when choosing a library.

[ Parent | on HN ]

Re:Show HN: Franc – Detect natural languages by wooorm on Friday October 03, 2014 @08:44AM (#8405241)

Agreed :)

[ Parent | on HN ]

Re:Show HN: Franc – Detect natural languages by allan_s on Friday October 03, 2014 @09:32AM (#8405431)

for the case of "one sentence detection" you can use Tatoeba project database dump http://tatoeba.org/eng/downloads

you have a CSV of iso code => sentence , which should be 99% accurate (as it gets user proofed), so on in which you can compare your tool with.

I think for longer text one could use Wikipedia dump or alike ?

[ Parent | on HN ]

Re:Show HN: Franc – Detect natural languages by michaelmior on Friday October 03, 2014 @09:56AM (#8405536)

Thanks for the pointer. I might decide to whip something up one of these days. I really have no need for language detection, but I just find it interesting and I'm curious to see wich libraries will win out.

[ Parent | on HN ]