Re:Code Page 437 Refuses to Die
by Can_Not on Sunday May 08, 2016 @11:16AM
(#11654358)
I've noticed that Excel seems to assert that CSVs are not in UTF8. I know the standard is actually ASCII, but nobody on the team seemed willing to understand why their smart quotes came out as "weird characters" (or even understand what smart quotes are). It came out correctly in LibreOffice.
[ Parent
| on HN ]
Re:Code Page 437 Refuses to Die
by TAForObvReasons on Sunday May 08, 2016 @11:42AM
(#11654465)
Excel predates Unicode by a few years. The first true Excel version on Mac was released in 1985. The first generally recognized use of Unicode dates back to 1988.
Excel CSVs default to the system codepage when reading and writing, which makes some sense given that the files are plain text. You can actually control what codepage is used when importing data but you have to use the import option.
This is why I always write a BOM in CSV files (yes, even though it's UTF-8 and it shouldn't have a BOM). Excel will actually respect it.
The bigger problem is when Excel tries to be helpful and reformats the data upon opening the file. I can't remember the intricacies, but it used to behave differently whether you opened it from a local path or directly from a browser. I seem to recall using an "incorrect" MIME type helped with that.
Adding a BOM to a CSV file then causes many libraries to add the BOM to the first column's heading name.
It's an option that just defaults to the system ANSI codepage. After clicking Data > From Text, in the "Text Import Wizard - Step 1 of 3" screen, change the "File origin" to "65001: Unicode (UTF-8)"