Re:Show HN: I made a modern web UI for Wikipedia
by dsizzle on Monday December 06, 2021 @03:59PM
(#29465630)
Interesting. What are the bandwidth considerations for this? I had the impression that many ISPs make self-hosting impractical on normal household plans, but then I've never looked into it seriously.
Re:Show HN: I made a modern web UI for Wikipedia
by marginalia_nu on Monday December 06, 2021 @04:11PM
(#29465788)
Dunno, I have unlimited uploads and downloads @ 100 mbit, and I host both this Wikipedia mirror as well as a search engine and some other stuff.
The bandwidth usage is pretty inconsequential since most of my pages clock in at just a few kilobytes.
My mirror clocks in at 21 Gb. I'm self hosting so there's no real money cost besides the one-time investment in hardware. A really chap person could probably host this off a raspberry pi attached to something like a Corsair Voyager and serve multiple requests per second.
There's some technical trickiness in actually storing the files in a file system, since most filesystems don't deal well with having millions of files in the same folder. So I've had to build a four-tier directory structure based on the hash of the file name to be able to store the files. They're stored in a structure like 31/444/781/225/foobar.gz.
I did some experiments storing them as BLOBs in a database, but I couldn't get it to work.