Alleged M4 MacBook Pro Unboxing Video Reveals These Four Upgrades

Re:Alleged M4 MacBook Pro Unboxing Video Reveals These Four ... by boroboro4 on Monday October 07, 2024 @08:29AM (#41765330)

There are different ways to run LLMs on multiple GPUs, one of them (called tensor parallelism) in low batch scenarios would be multiplying bandwidth between different GPUs. So no, 8 4090s is not 1000 GB/s.

[ Parent | on HN ]

Re:Alleged M4 MacBook Pro Unboxing Video Reveals These Four ... by klohto on Monday October 07, 2024 @08:48AM (#41765486)

let me know how is the PCIe bandwidth treating you

[ Parent | on HN ]

Re:Alleged M4 MacBook Pro Unboxing Video Reveals These Four ... by boroboro4 on Tuesday October 15, 2024 @03:40PM (#41852327)

Since we’re talking about small batch sizes PCIe bandwidth isn’t as important - intermediate hidden state is magnitude smaller than weights.

[ Parent | on HN ]

Re:Alleged M4 MacBook Pro Unboxing Video Reveals These Four ... by behnamoh on Monday October 07, 2024 @07:24PM (#41772188)

you've heard something and are regurgitating it without fully understanding it.

[ Parent | on HN ]

Re:Alleged M4 MacBook Pro Unboxing Video Reveals These Four ... by boroboro4 on Tuesday October 15, 2024 @03:38PM (#41852306)

I’m developing inference engine, so I actually do understand how it works. As well as other types of parallelism and how exactly they do different trade offs

[ Parent | on HN ]