News for hackers. Stuff that matters.
Re:CleverCSV: A Drop-In Replacement for Python's CSV Module by scrollaway on Monday January 13, 2020 @06:21PM (#22039886)
I knew of ftfy already but I would love some more examples of these kinds of data cleanup libraries; it's always super useful.
Re:CleverCSV: A Drop-In Replacement for Python's CSV Module by cosmie on Monday January 13, 2020 @11:52PM (#22041957)
Not in any particular order, but here is a list of really useful text butchering utilities, that I keep in the same grab-bag as ftfy: csvclean[1], unidecode[2], transliterate[3], charset-normalizer[4], cchardet[5], phonenumbers[6][7], iconv[8], enca/enconv[9].

[1] https://csvkit.readthedocs.io/en/1.0.3/scripts/csvclean.html

[2] https://pypi.org/project/Unidecode/

[3] https://pypi.org/project/transliterate/

[4] https://pypi.org/project/charset-normalizer/

[5] https://pypi.org/project/cchardet/

[6] https://pypi.org/project/phonenumbers/

[7] which is a python port of Google's libphonenumber: https://github.com/google/libphonenumber

[8] https://linux.die.net/man/1/iconv

[9] https://linux.die.net/man/1/enca

Re:CleverCSV: A Drop-In Replacement for Python's CSV Module by joshvm on Tuesday January 14, 2020 @07:31AM (#22044168)
This software (Trifacta Wrangler) was demoed at NeurIPS this year and looked very impressive. https://www.trifacta.com/products/wrangler-editions

There is also the non-maintaned version that the company spun out from:

http://vis.stanford.edu/wrangler/