Yeah this big time. I haven’t found a solution that makes sense. Larger models are already good enough and so convenient.

When it’s more feasible to do inference on the client (browser or desktop) I can see SLMs popping up more common in production.