Re:Tensorbook
by sigmoid10 on Wednesday December 28, 2022 @05:53AM
(#34159662)
This thing is absurd. I work in ML research and I use my Macbook Air M1 to train LLMs from anywhere, but I would never let them touch my local machine. Even if they'd run (most of them wouldn't - even on a 3080 Ti), they would ruin my battery life. So I simply connect to a GPU server via VPN and ssh + vs code remote. Only students or beginner enthusiasts might think that training this stuff on the go would actually need a beefy laptop.
[ Parent
| on HN ]
Is it though? I personally prefer local development and test out my code instantly, especially when internet is flaky. Once you have confidence in your code, you train it in the cloud.
Develop on a tiny slice of you dataset locally of you must. Then train properly elsewhere (in the cloud of your choice)
You could do this via Jupiter notebooks and visual studio code.
Re:Tensorbook
by goethes_kind on Wednesday December 28, 2022 @09:27AM
(#34161258)
I develop my code directly on the HPC cluster. What do you even mean with testing your code instantly? You don't have to run a batch job every time you need to test something. You can use interactive mode.
You will probably keep your dataset on Cloud storage and Google Cloud has much higher Network speed, it makes things much faster than working locally
Obviously if you have your own local data center then you would use that
I can always test out my code instantly. With the vs code remote ssh plugin it's exactly like running everything locally, but in reality only the GUI runs on my laptop. The necessary connection speeds are tiny compared to what you'd need on the go to sync models and datasets.
Transformer adapters/fine-tuning would be a good fit, especially while traveling with a power source next to you.
This product is AI/ML equivalent or "gaming" or "military grade" products, all of which are just over-priced crap with RGB/Camo added.
Re:Tensorbook
by mark_l_watson on Wednesday December 28, 2022 @08:52AM
(#34160928)
I agree with you. I bought an at home GPU rig 5 years ago, and it was good for a while. Then I realized how much time I could save myself by paying Google Colab $10/month for improved GPU access - close to zero config hassles. Running in Jupiter style notebooks is not optimal, but it is good enough. I am in my 70s and maybe it is because of my age, but I look critically at all my activities and always ask “is this really what I want to be spending time in today?”
> I am in my 70s
You sir are very inspiring
By that reasoning, why does the M1 even need a GPU?
M1 chip is both a CPU/GPU, all computers need at least a limited GPU
You need some sort of GPU to run any OS in a decent way