News for hackers. Stuff that matters.

A comment on Serving 70B-Scale LLMs Efficiently on Low-Resource Edge Devices [pdf]. Parent comment

Re:Serving 70B-Scale LLMs Efficiently on Low-Resource Edge D... by adam_arthur on Thursday October 03, 2024 @03:21PM (#41733997)
Not sure what you mean: https://aws.amazon.com/ec2/graviton/

Not to speak of managed cloud services that run on ARM under-the-hood/behind the scenes.

Of course ARM isn't inherently cheaper, AMD+Intel could cut prices/margins big and probably be competitive on $/perf

[ Parent | on HN ]
Re:Serving 70B-Scale LLMs Efficiently on Low-Resource Edge D... by lxgr on Friday October 04, 2024 @09:51AM (#41741403)
That's what I mean: ARM was initially attractive in the low-power/low-performance market, then it scaled up to higher and higher power cores (while still staying power efficient), which in turn attracted datacenter customers.
[ Parent | on HN ]