News for hackers. Stuff that matters.

A comment on Anthropic - Introducing 100K Token Context Windows, Around 75,000 Words. Parent comment

Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by mejutoco on Thursday May 11, 2023 @02:55PM (#35906644)
I am confused. Those amounts are ram, not gpu ram, aren‘t they? Macs cpus are impressive, but not for ml. A most realistic one for a consumer is a 4090 rtx 24 GB. A lot of models do not fit in that, so A6000 48GB and over for some professional cards. That might be around 9000€ already.
[ Parent | on HN ]
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by codedokode on Thursday May 11, 2023 @03:55PM (#35907589)
> Macs cpus are impressive, but not for ml

On Mac GPU has access to all memory.

[ Parent | on HN ]
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by piperswe on Thursday May 11, 2023 @03:58PM (#35907614)
Apple Silicon has unified memory - all memory is accessible to both the CPU and GPU parts of the SoC.
[ Parent | on HN ]
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by karmasimida on Thursday May 11, 2023 @04:36PM (#35908041)
But they comes at max 32GB model?
[ Parent | on HN ]
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by mkl on Thursday May 11, 2023 @05:48PM (#35908849)
Mac Studio (desktop) is up to 128GB, and Macbook Pro is up to 96GB.
[ Parent | on HN ]