Re:Train Your Machine Learning Models on Google’s GPUs for Free
by londons_explore on Thursday March 15, 2018 @02:47PM
(#16595477)
The memory limitation is on the host (ya know - the stuff you don't actually need). You get 12GB of GPU memory I think.
You can mount a remote ssh disk as a working directory, and for your actual data stream it in from cloud storage direct from tensorflow.
Re:Train Your Machine Learning Models on Google’s GPUs for Free
by singularity2001 on Thursday March 15, 2018 @02:44PM
(#16595466)
[[deleted]]
Re:Train Your Machine Learning Models on Google’s GPUs for Free
by londons_explore on Thursday March 15, 2018 @02:47PM
(#16595486)
They give you 12GB of GPU ram.
The 500mb is simply the host ram for the python runtime. As long as you aren't converting everything to numpy arrays, you'll be fine.
Re:Train Your Machine Learning Models on Google’s GPUs for Free
by singularity2001 on Thursday March 15, 2018 @02:50PM
(#16595514)
fantastic, thanks!
First, is memory. I’ve seen reports of practical limitations being around 500mb. Call me a spoiled millennial noob but the vast majority of my real-world ML projects use a lot more - especially when working with image data.
Second, is getting data into your notebook. Most ML is data heavy and you want a fast way of working with it. For example, the Google Landmark challenge on Kaggle has nearly half a terabyte of unaugmented images just for the test set. You could easily push several terabytes for that one-off challenge alone if you were not careful.
Note: it is possible to work with much less data in many cases by resizing and preprocessing the images as you pull them down but that has problems too. Also, you can use Google Drive but it’s not ideal for large datasets.
Related:
https://www.reddit.com/r/MachineLearning/comments/84532y/n_g...
https://stackoverflow.com/questions/48750199/google-colabora...