Right, so it can't be used on proprietary code or in settings where personal data might occur.
To be honest, I didn’t realize that model selection would be such an important point for users. I believed that choosing a high-quality model with strong reasoning capabilities was part of the service’s value proposition.
But lately, more users — including yourself — have been asking for support for other models like Claude Sonnet or LLaMA.
I’m now seriously considering adding an adapter feature. Thank you for your feedback — I really appreciate it.
Together with the CEO I've also decided that we do not do this with our own code, it stays on machines we control until someone pays for some artifact we'd like to license.
I'm well aware that many other organisations take a different position and push out basically everything they work on to SaaS LLM:s, in my experience defending it with something about so called productivity and something about some contract clause about the SaaS pinky promising to not straight up take the code. But nothing stops them from running hidden queries against it with their in-house models parallel with providing their main service, and sift out a lot of trade secrets and other goodies from it.
It's also likely these SaaS corporations can benchmark and otherwise profile individual developers, information that would be very valuable to e.g. recruiting agencies.
If you can tell, how is that Copilot performance measured?
I don't know how they measure it, but I assume they have some management dashboard they can view. The requirement is exposed to us in multiple performance objectives with phrases about x% of team making active use of gen ai, use gen ai to increase velocity y%, automate z% of tasks with gen ai, and so on.
2. from now, i only allow to use gpt-4o, because the requests involve relatively long context windows, which require high-quality reasoning. Only recent high-performance models like GPT-4o or Claude Sonnet are capable of reducing the manual workload for this kind of task.
___
but still, if user want to use other models , i can make adapter features for various models