News for hackers. Stuff that matters.
Re:ASML EUV Lithography Machine Could Keep Moore’s Law on Track by rodoxcasta on Saturday July 29, 2023 @10:18PM (#36927133)
For inference, at least locally, the bottleneck is usually the memory bandwidth (and quantity, of course).

I hope that AI hype lead us to more memory and more memory bandwidth, because they are really lagging behind computer power increase from like 15 years already.

Re:ASML EUV Lithography Machine Could Keep Moore’s Law on Track by sbrother on Sunday July 30, 2023 @12:09AM (#36927725)
Oh, 100%. But you can do some pretty amazing things with fine-tuning LLMs too, and that is very compute intensive. Not to mention it's ridiculously hard even getting access to a cloud GPU instance nowadays.