This is the frontier—tiny, specialized models like this and ReALM [0], coupled to the application logic and able to run on-device.

Eventually devices will be powerful enough to run more general purpose models locally, but for high-frequency user tasks with a low tolerance for error, smaller specialized models may always win.

[0]: https://arxiv.org/abs/2403.20329