Books in .txt format for AI training purposes

Re:Books in .txt format for AI training purposes by gilrain on Sunday October 25, 2020 @12:39PM (#24887566)

What is ".txt format"? I'm genuinely curious what they went with here, but don't want to download 36 GB to find out. There isn't a .txt standard, is there?

I would hope for UTF-8, but given the old-school, Windows-y .txt extension, it could just as easily be Windows-1252 or something. LF or CRLF line endings?

[ Parent | on HN ]

Re:Books in .txt format for AI training purposes by SanchoPanda on Sunday October 25, 2020 @04:05PM (#24889452)

It looks like they are unix line endings, UTF-8, and markdown formatted.

[ Parent | on HN ]

Re:Books in .txt format for AI training purposes by gilrain on Sunday October 25, 2020 @04:58PM (#24889829)

Thanks!

[ Parent | on HN ]