News for hackers. Stuff that matters.
Re:Running PyTorch on the M1 GPU by londons_explore on Saturday May 21, 2022 @07:55AM (#31457105)
What ML are people actually wanting to do on M1 laptops?

I was under the impression that outside toy problems, you can't really get decent results unless you have a TPU or fleet of GPU's for training. If you just train on a laptop you're shooting yourself in the foot compared to someone who gets the same job done 10x faster in the cloud (usually with free credits).

Obviously inference is still possible on a laptop for many models... but not many 'hacker' tasks involve only inference.

Re:Running PyTorch on the M1 GPU by visionscaper on Saturday May 21, 2022 @09:14AM (#31457592)
I’m not using an M1 machine yet, but I think is extremely useful to be able to do some small scale experiments for testing and debugging purposes locally before starting a full-scale training job in the cloud.

Further, in general, I’m really charmed by the potential of having unified memory; the idea you can test some batch training iterations with a batch size that fills almost all 128MB is a unique capability!

Last, a large amount of unified memory allows to do inference and “prompt engineering” with very large models, locally. E.g. Using GPT-J like models (6B parameters).

Re:Running PyTorch on the M1 GPU by danieldk on Saturday May 21, 2022 @09:44AM (#31457846)
What ML are people actually wanting to do on M1 laptops?

(1) development - quickly train smaller networks locally to validate the code. Then offload to a big GPU machine. (2) Smaller convolutional networks and RNNs may be below SOTA, but can still provide a good performance-accuracy trade-off. You could already train these networks pretty fast on an M1 thanks to AMX, but his makes training such networks even faster.

Re:Running PyTorch on the M1 GPU by Tenoke on Saturday May 21, 2022 @08:52AM (#31457411)
>I was under the impression that outside toy problems, you can't really get decent results unless you have a TPU or fleet of GPU's for training.

This is most certainly not the case. It's impossible to achieve SOTA on most benchmarks but there's a lot more ML and data analysis done on smaller datasets where you don't particularly need a fleet of GPUs.

Re:Running PyTorch on the M1 GPU by andybak on Saturday May 21, 2022 @08:20AM (#31457231)
The whole Diffusion text to image thing which is hugely popular right now doesn't involve any training. Same for text generation where you download pre-trained models. I'm using a laptop with a 3080 and getting much faster results than I typically get from a paid Colab account (not Pro mind you).
Re:Running PyTorch on the M1 GPU by natly on Saturday May 21, 2022 @01:26PM (#31459877)
Which repo/tool are you using for local text to image diffusion if you don't mind sharing? :)
Re:Running PyTorch on the M1 GPU by andybak on Saturday May 21, 2022 @01:37PM (#31460007)
Not at all. I discovered that Visions of Chaos (originally developed for fractals etc) has extensive support for installing and running ML models

https://softology.pro/tutorials/tensorflow/tensorflow.htm

It handles (most) of the installation issues and gives you a GUI. You can however bypass the GUI. Every time you run a command the console shows you what the command line looks like so copy/paste into a terminal and away you go.

Re:Running PyTorch on the M1 GPU by arecurrence on Saturday May 21, 2022 @11:59AM (#31459003)
There's plenty of training related stuff that 10 TFlops is sufficient for. Fine tuning models for example is a great use case for a high ram yet relatively low TFlops environment.

I don't know how much you've done with running ml jobs on cloud boxes but it can be very annoying to work with them when you are actively iterating. Especially if you are doing a lot of work outside the cloud box environment. Even turn key consumer products like Colab can fall over if you accidentally press "Back" in your browser... I once lost many hours of work because I inadvertently swiped left :)

Apple will continue improving their performance in this space and once jobs that run for hours can be run overnight... I suspect people will often choose to run overnight simply for the ergonomics benefit.

Re:Running PyTorch on the M1 GPU by shimmeringleaf on Saturday May 21, 2022 @08:19AM (#31457221)
Totally, performance for actual useful training etc. is at least a couple of hardware generations away. This is super useful however for people like me who like to use an m1 Mac as their daily driver and don’t always want to have to fire up a remote rig just to sketch a little. Also being able to do some local inference can be great for when building a web GUI, etc.

So absolutely not mature, but having PyTorch working better also on macs is only positive imho.

Re:Running PyTorch on the M1 GPU by nl on Saturday May 21, 2022 @08:58AM (#31457460)
I've fine tuned transformers on CPUs before for weird reasons. Sometimes the computer you have is what you make do with.
Re:Running PyTorch on the M1 GPU by singularity2001 on Saturday May 21, 2022 @10:53AM (#31458412)
Ideally fine-tuning foundation models. Training them costs several millions, so out of scope no matter which GPU you have.