Apple M3 Ultra - Hackdot

Re:Apple M3 Ultra by staticman2 on Wednesday March 05, 2025 @01:03PM (#43270035)

Smaller, dumber models are faster than bigger, slower ones.

What model do you find fast enough and smart enough?

[ Parent | on HN ]

Re:Apple M3 Ultra by Matl on Wednesday March 05, 2025 @02:42PM (#43271297)

Not OP but I am finding the Qwen 2.5 32b distilled with DeepSeek R1 model to be a good speed/smartness ratio on the M4 Pro Mac Mini.

[ Parent | on HN ]

Re:Apple M3 Ultra by a1o on Wednesday March 05, 2025 @06:55PM (#43274442)

How much RAM?

[ Parent | on HN ]

Re:Apple M3 Ultra by Matl on Thursday March 06, 2025 @04:19PM (#43285169)

It takes between 22GB-37GB depending on the context size etc. from what I've observed.

[ Parent | on HN ]

Re:Apple M3 Ultra by a1o on Friday March 07, 2025 @08:33AM (#43290007)

Thanks!

[ Parent | on HN ]

Re:Apple M3 Ultra by bastardoperator on Thursday March 06, 2025 @11:44AM (#43282232)

I'm running the same exact models.

[ Parent | on HN ]