Re:AI Dungeon 2 costing over $10k/day to run on GCS/Colab
by kbenson on Monday December 09, 2019 @04:01AM
(#21741287)
This is sort of a killer example for the layperson of what ML can do, so hopefully Google will recognize this and comp most/all the data egress since it's a drop in the bucket for them, but every person that uses it can still go "Wow, Google's services allows for some amazing stuff."
Re:AI Dungeon 2 costing over $10k/day to run on GCS/Colab
by Piskvorrr on Monday December 09, 2019 @05:13AM
(#21741538)
Oh, it does. NOT.FOR.FREE, though; that's the entire reason it exists at all.
In other words, yes, this is an amazing amount of raw power - with a corresponding price tag.
Re:AI Dungeon 2 costing over $10k/day to run on GCS/Colab
by krick on Monday December 09, 2019 @07:35AM
(#21742058)
But you could set up your own file-serving node and then data transfer to end-user Colab instances would be free, right? ("Free" — costing only as much your CDN costs for 360TB/day, which is still quite pricey, I think, but not as much as $10K/day. I.e. Google wouldn't charge you for data transfer here.)
Re:AI Dungeon 2 costing over $10k/day to run on GCS/Colab
by dodobirdlord on Tuesday December 10, 2019 @03:31AM
(#21750714)
Yup. I think I saw yesterday that they were looking to move the model to BitTorrent.
60k users yesterday * 6GB each -> 360TB of data egress!
Normally, a scenario like this wouldn't involve bandwidth costs because GCP -> GCP same-region bandwidth is free, but Colab is technically not part of GCP, so the bandwidth charge is being assessed as egress to the public internet, which is pricy for that much data. Though it's probably still a lot cheaper than paying for the GPU-hours for that many users.