News for hackers. Stuff that matters.
Re:Tensorbook by sigmoid10 on Wednesday December 28, 2022 @05:53AM (#34159662)
This thing is absurd. I work in ML research and I use my Macbook Air M1 to train LLMs from anywhere, but I would never let them touch my local machine. Even if they'd run (most of them wouldn't - even on a 3080 Ti), they would ruin my battery life. So I simply connect to a GPU server via VPN and ssh + vs code remote. Only students or beginner enthusiasts might think that training this stuff on the go would actually need a beefy laptop.
Re:Tensorbook by whazor on Wednesday December 28, 2022 @06:21AM (#34159883)
Is it though? I personally prefer local development and test out my code instantly, especially when internet is flaky. Once you have confidence in your code, you train it in the cloud.
Re:Tensorbook by dankle on Wednesday December 28, 2022 @07:20AM (#34160279)
Develop on a tiny slice of you dataset locally of you must. Then train properly elsewhere (in the cloud of your choice)
Re:Tensorbook by ekleraki on Wednesday December 28, 2022 @08:15AM (#34160641)
You could do this via Jupiter notebooks and visual studio code.
Re:Tensorbook by goethes_kind on Wednesday December 28, 2022 @09:27AM (#34161258)
I develop my code directly on the HPC cluster. What do you even mean with testing your code instantly? You don't have to run a batch job every time you need to test something. You can use interactive mode.
Re:Tensorbook by atroxone on Wednesday December 28, 2022 @12:37PM (#34163466)
You will probably keep your dataset on Cloud storage and Google Cloud has much higher Network speed, it makes things much faster than working locally

Obviously if you have your own local data center then you would use that

Re:Tensorbook by sigmoid10 on Wednesday December 28, 2022 @07:42PM (#34168042)
I can always test out my code instantly. With the vs code remote ssh plugin it's exactly like running everything locally, but in reality only the GUI runs on my laptop. The necessary connection speeds are tiny compared to what you'd need on the go to sync models and datasets.
Re:Tensorbook by bitL on Wednesday December 28, 2022 @07:58AM (#34160528)
Transformer adapters/fine-tuning would be a good fit, especially while traveling with a power source next to you.
Re:Tensorbook by anaganisk on Wednesday December 28, 2022 @08:31AM (#34160766)
This product is AI/ML equivalent or "gaming" or "military grade" products, all of which are just over-priced crap with RGB/Camo added.
Re:Tensorbook by mark_l_watson on Wednesday December 28, 2022 @08:52AM (#34160928)
I agree with you. I bought an at home GPU rig 5 years ago, and it was good for a while. Then I realized how much time I could save myself by paying Google Colab $10/month for improved GPU access - close to zero config hassles. Running in Jupiter style notebooks is not optimal, but it is good enough. I am in my 70s and maybe it is because of my age, but I look critically at all my activities and always ask “is this really what I want to be spending time in today?”
Re:Tensorbook by victor106 on Wednesday December 28, 2022 @09:41AM (#34161382)
> I am in my 70s

You sir are very inspiring

Re:Tensorbook by amelius on Wednesday December 28, 2022 @08:55AM (#34160949)
By that reasoning, why does the M1 even need a GPU?
Re:Tensorbook by levesque on Wednesday December 28, 2022 @10:04AM (#34161615)
M1 chip is both a CPU/GPU, all computers need at least a limited GPU
Re:Tensorbook by atroxone on Wednesday December 28, 2022 @12:38PM (#34163481)
You need some sort of GPU to run any OS in a decent way
Re:Tensorbook by sigmoid10 on Wednesday December 28, 2022 @07:48PM (#34168086)
You couldn't even watch modern youtube videos on most cpus were it not for integrated accelerators. The M1 does exactly that, but in a power efficient and reasonably performant way.