News for hackers. Stuff that matters.

A comment on Apple introduces M4 chip. Parent comment

Re:Apple introduces M4 chip by whimsicalism on Tuesday May 07, 2024 @07:18PM (#40292628)
Having used groq and other fast LLM services a fair bit, lag seems negligible. You're literally just passing text at close to the speed of light.
[ Parent | on HN ]
Re:Apple introduces M4 chip by jonplackett on Tuesday May 07, 2024 @07:50PM (#40292856)
* when you have a good internet connection

** when you live in the USA

[ Parent | on HN ]
Re:Apple introduces M4 chip by whimsicalism on Tuesday May 07, 2024 @08:06PM (#40292944)
> * when you have a good internet connection

Or at least, a good enough internet connection to send plaintext.

> * when you live in the USA

Even from Australia to USA is just ~300ms of latency for first token and then the whole thing can finish in ~1s. And making that faster doesn't require on-device deployment, it just requires a server in Australia - which is obviously going to be coming if it hasn't already for many providers.

[ Parent | on HN ]