Apple MacBook Air 15-Inch M3 Review

Re:Apple MacBook Air 15-Inch M3 Review by seanmcdirmid on Monday April 29, 2024 @12:35AM (#40194483)

I think on-device models would be really useful. Imagine a conversational interface with much less latency so the conversations felt real. I wonder what kind of computing power we will need before we get there (e.g. running an LLM with lots of prompt data + on device speech recognition), maybe 5-10 years?

[ Parent | on HN ]

Re:Apple MacBook Air 15-Inch M3 Review by Shish2k on Monday April 29, 2024 @10:08AM (#40198538)

> Imagine a conversational interface with much less latency

With current models the latency comes from processing, not from the network — going from a high-power remote server to a low-power local phone is likely to increase latency more than it reduces it

[ Parent | on HN ]

Re:Apple MacBook Air 15-Inch M3 Review by ec109685 on Monday April 29, 2024 @12:13PM (#40200243)

It depends on how big the model is. They are using a little LLM to help correct text and that would be dreadful if it took a server round trip.

[ Parent | on HN ]