News for hackers. Stuff that matters.
Re:Show HN: Free database of geographic place names and geos... by Quai on Friday September 25, 2015 @06:21AM (#10277248)
First three norwegian cities I looked up was given the wrong name. (Plesund (should be Ålesund), Bodi (Bodø) and Tdnsberg (Tønsberg)). So, I think it is safe to say that there are encoding issues in the data set.
Re:Show HN: Free database of geographic place names and geos... by bhaak on Friday September 25, 2015 @06:28AM (#10277267)
There's definitely something wrong.

Looking at the SQL file, "Zürich" is listed as "Zdrich", Munich is only shown as "Munich" without the German name "München" anywhere, and Cologne is listed as "Cologne" with the wrong "Koln" as alternative name.

Doesn't look like a reliable data source.

Re:Show HN: Free database of geographic place names and geos... by seszett on Friday September 25, 2015 @06:40AM (#10277313)
French names have the same problem, with "La Réunion" being shown as "La Rcunion" and "Rhône-Alpes" as "RhAne-Alpes".

It's strange, I can't really understand how these accents can get mangled as these particular single letters.

Re:Show HN: Free database of geographic place names and geos... by vcarl on Friday September 25, 2015 @09:31AM (#10278007)
Almost looks like OCR errors, especially Rhône -> RhAne.
Re:Show HN: Free database of geographic place names and geos... by bhaak on Friday September 25, 2015 @09:48AM (#10278087)
I don't think that the reason are OCR errors. It's too consistent for that.

I think I also have seen such character set corruptions before, but I can't remember how exactly you can get these particular corruptions.

Re:Show HN: Free database of geographic place names and geos... by marco1 on Friday September 25, 2015 @06:19PM (#10281058)
They're not OCR errors, unfortunately. Sorry for these problems! We have to investigate!
Re:Show HN: Free database of geographic place names and geos... by maze-le on Friday September 25, 2015 @06:44AM (#10277341)
It's 2015 and we still have UTF-8 problems... sigh
Re:Show HN: Free database of geographic place names and geos... by spacemanmatt on Friday September 25, 2015 @10:07AM (#10278200)
Or this is just a sloppy translation/compilation from better sources.
Re:Show HN: Free database of geographic place names and geos... by marco1 on Friday September 25, 2015 @06:18PM (#10281056)
Sorry for the inaccuracies!

We'd love to fix all those issues -- maybe with some help from the community :)

The goal is definitely to turn this into a more and more reliable data source every day.

Re:Show HN: Free database of geographic place names and geos... by marco1 on Friday September 25, 2015 @06:21PM (#10281070)
This is definitely a major problem that has to be fixed! Thanks for pointing this out again.

The complete build process is described in "SOURCE.md". That's where we got these encoding issues from -- right from the source material. Maybe we didn't open or parse the source material correctly. But we were not able to get them without the encoding errors.

We'll love to fix all these issues :)

Re:Show HN: Free database of geographic place names and geos... by lobster_johnson on Friday September 25, 2015 @07:16PM (#10281302)
Instead of Markdown file, you should consider creating a set of scripts that will fetch, cache and process upstream data into the desired formats. As someone interested in using this data, I really need to be able to easily run the entire build process myself.
Re:Show HN: Free database of geographic place names and geos... by marco1 on Saturday September 26, 2015 @07:24AM (#10282773)
Of course, you're right, thanks! This has been noted down as one of the next steps that are required.