Re:Apple introduces M4 chip
by whimsicalism on Tuesday May 07, 2024 @08:06PM
(#40292944)
> * when you have a good internet connection
Or at least, a good enough internet connection to send plaintext.
> * when you live in the USA
Even from Australia to USA is just ~300ms of latency for first token and then the whole thing can finish in ~1s. And making that faster doesn't require on-device deployment, it just requires a server in Australia - which is obviously going to be coming if it hasn't already for many providers.
Or at least, a good enough internet connection to send plaintext.
> * when you live in the USA
Even from Australia to USA is just ~300ms of latency for first token and then the whole thing can finish in ~1s. And making that faster doesn't require on-device deployment, it just requires a server in Australia - which is obviously going to be coming if it hasn't already for many providers.