Re:Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups –...
by MandieD on Monday December 09, 2019 @03:55PM
(#21746451)
I’m running the Docker image on the smallest Hetzner VMs, with 5 concurrent groups and 40 shared rsync threads per container, and 12 containers per server. Start one container, do docker top on it to make sure it’s pulling, then start the others one by one, taking a few seconds between each to avoid overwhelming the CPU. I’ve got 6 of those little VMs going, and have rolled up 4GB and 2800 groups worth in 6 hours.
After they settle down, they’re more memory than processor intensive. I’ve considered playing with the settings a bit, but thought it was more important to get a bunch of them running on a couple different VMs at different sites.
If I were really feeling fancy, I’d write a nice deployment definition for orchestrating this with microk8s...
Re:Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups –...
by kalleboo on Sunday December 08, 2019 @11:39PM
(#21740436)
I'm running it on a Synology NAS (Celeron J3455), and the docker manager UI claims it's using 180 MB RAM and less than 1% CPU (and I just confirmed it's currently working on archiving Yahoo! Groups)
Re:Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups –...
by icebraining on Sunday December 08, 2019 @07:28PM
(#21739516)
I don't find it processor or memory heavy, it's mostly doing a lot of IO (network and disk).
What are the hardware requirements of that VM? I'm attempting to import it on my NAS4Free home NAS Virtualbox service which is the only machine I keep up 24/7 atm, but it takes forever to import. The hardware is very limited however (Atom D410 + a bit over 1GB RAM available), so I'm not sure it would succeed, but so far it loads forever, no errors given. I'd like to run it for this project to start contributing quickly albeit with limited hw before the deadline, then find better iron in the future.