OpenAI Negotiations to Reinstate Altman Hit Snag over Board Role

Re:OpenAI Negotiations to Reinstate Altman Hit Snag over Boa... by q7xvh97o2pDhNrh on Sunday November 19, 2023 @09:49PM (#38341630)

That's fascinating that the total size is so tiny — only 2.4 TB‽

I assume this must be only the text portion, and heavily compressed?

[ Parent | on HN ]

Re:OpenAI Negotiations to Reinstate Altman Hit Snag over Boa... by lxgr on Sunday November 19, 2023 @10:30PM (#38341937)

Text really doesn't take up that much space, and in addition it compresses pretty well.

The entire English language Wikipedia is only around 60GB in a format that can be readily searched and randomly accessed (ZIM), for example: https://kiwix.org/

[ Parent | on HN ]

Re:OpenAI Negotiations to Reinstate Altman Hit Snag over Boa... by lmm on Sunday November 19, 2023 @10:39PM (#38341995)

Does Kiwix actually work? I see people hyping it here but I could never get it to actually, y'know, download the file and display the wikipedia on my phone.

[ Parent | on HN ]

Re:OpenAI Negotiations to Reinstate Altman Hit Snag over Boa... by lxgr on Sunday November 19, 2023 @10:42PM (#38342012)

It works perfectly for me, both on iOS and macOS.

[ Parent | on HN ]

Re:OpenAI Negotiations to Reinstate Altman Hit Snag over Boa... by pc2slow4webpack on Sunday November 19, 2023 @10:55PM (#38342073)

Just downloaded. Doesn't seem to want to download Wikipedia on my phone, it says "detecting if filesystem supports 4gb files"

[ Parent | on HN ]

Re:OpenAI Negotiations to Reinstate Altman Hit Snag over Boa... by vatueil on Sunday November 19, 2023 @10:58PM (#38342086)

Kiwix worked for me. IIRC there may be difficulties opening an archive that was downloaded outside of the mobile app, but archives downloaded in-app were fine.

For the mobile app I used one of the smaller Wikipedia subsets, since I didn't want to take up too much space on my phone. The full offline Wikipedia download is saved to my laptop.

[ Parent | on HN ]