This has led to developers attempting to conflate the two distinctly different concepts and make APIs support both while behaving differently.
A simple solution is there in plain sight: just use exclusively b"" and u"" strings for any code you wish to work in both Py2 and Py3, and forget about "". All and any libraries should be using those exclusively if they support both. Python3-only code should be using b"" and "" instead.
One could consider this a design oversight in Python 3: the fact that the syntax is so similar elsewhere makes people want to run the same code in both, yet a core type is basically incompatible.
from __future__ import unicode_literals
This has not been the case since 2012. The last release of Python 3 for which this was the case reached end of life in February 2016. Please stop misinforming people.
It's absolutely not correct, because there are many APIs which take "native strings" across versions aka they take an `str` (byte string) in Python 2 and an `str` (unicode string) in Python 3. unicode_literals causes significantly more problems than they solve.
The correct cross-platform solution (and the very reason why the `u` prefix was reintroduced after having initially been removed from Python 3) is in fact to use b"" for known byte strings, u"" for known text, and "" for "native strings" to interact with APIs which specifically need that.
I've not really had many other issues porting from 2 -> 3 from my own code; issues usually arise from 3rd-party libs that are relied upon (especially if they utilize C/C++ extensions). DB libs have sometimes been problematic. IIRC, pysybase requires you to set the encoding of strings now (which wasn't required before). I use pysybase to talk to both Sybase & MS SQL (it talks the TDS protocol).
???
The Python 3 CSV module works on text only (unicode), and you seem to be misreading the note: the module does in fact do newline transparency: https://docs.python.org/3/library/csv.html#id3
`newline=''` is to be specified on the file so it doesn't perform newline translation, because that would change the semantics of newlines in quoted strings: by default, `open` will universally translate newlines to `\n`, `newline=''` means it still does universal newlines (for purpose of iteration) but it doesn't do translation, returning newline sequences unchanged.
The Python 2 CSV module only worked on byte strings, and magic bytes.
It's better in some ways. But then, I gave up long ago on manipulating files with Python, because Python 3 simply decided that anything on my filesystem is utf-8 by default.
Want to get an stream out of a file? Better have utf-8 content. Otherwise all the documentation will be buried 15 meters deep, outdated on some aspects and dependent on some not-released yet features.
Want to name a file? Better have an utf-8 name. I gave-up on python before I was able to open a file with a name that doesn't conform to utf-8.
Want to read a line at a time? Sorry, we only do that to text. Go get an utf-8 file.
Want to match with a regular expression or parse it somehow? Sorry, we only do that to text.
And so on.
> Want to get an stream out of a file? Better have utf-8 content. Otherwise all the documentation will be buried 15 meters deep, outdated on some aspects and dependent on some not-released yet features.
f = open("myfile.txt", "r", encoding=ENCODING)
export NAME="$(head -c 5 /dev/urandom)"; touch "$NAME"; python -c 'import os; open(os.environ["NAME"])'
fh = open("foo", "rb"); lines=[l for l in fh]
import re; re.match(b"abc", b"abcdef")