News for hackers. Stuff that matters.
Re:It's OpenAI's world, we're just living in it by yubblegum on Friday October 10, 2025 @02:17PM (#45542063)
> Migrating to a different LLM is an afternoon's work at most, not nearly the complexity of porting an app between OS' or creating a robust hardware driver model.

I question this. Each vendor's offering has its own peculiar prompt quirks, does it not? Theoretically, switching RDBMS vendors (say Ora to Ingress) was also "an afernoon's work" but it never happened. The minutia is sticky with these sort of almost-but-not 'universal' interfaces.

Re:It's OpenAI's world, we're just living in it by mgh95 on Friday October 10, 2025 @02:21PM (#45542109)
> I question this. Each vendor's offering has its own peculiar prompt quirks, does it not? Theoretically, switching RDBMS vendors (say oracle to postgres) was also "an afernoon's work" but it never happened. The minutia is sticky with these sort of almost-but-not 'universal' interfaces.

The bigger problem is that there was never a way to move data between oracle->postgres in pure data form (i.e. point pgsql at your oracle folder and it "just works"). Migration is always a pain, and thus there is a substantial degree of stickiness, due to the cost of moving databases both in terms of risk and effort.

In contrast, vendors [1] are literally offering third party LLMS (such as claude) in addition to their own and offering one-click switching. This means users can try and if they desire switch with little friction.

[1] https://blog.jetbrains.com/ai/2025/09/introducing-claude-age...

Re:It's OpenAI's world, we're just living in it by RussianCow on Friday October 10, 2025 @04:11PM (#45543249)
Moreover, it's trivial to run several LLMs side-by-side for a while and measure the success of each, then migrate to the one that performs the best. And you can even migrate in-progress chats since all the context is passed on each call anyway.
Re:It's OpenAI's world, we're just living in it by gretch on Friday October 10, 2025 @03:53PM (#45543047)
The current LLMs support data export/import inherently because the interface is pure text.

All one needs to do is say something like “tell me all of personalization factors you have on me” and then just copy and paste that into the next LLM with “here’s stuff you should know about how to personalize output for me”

Re:It's OpenAI's world, we're just living in it by kllrnohj on Friday October 10, 2025 @02:38PM (#45542276)
> The minutia is sticky with these sort of almost-but-not 'universal' interfaces.

True, but that's not really applicable here since LLMs themselves are not stable, and are certainly not stable within a vendors own product line. Like imagine if every time Oracle shipped a new version it was significantly behaviorally inconsistent with the previous one. Upgrading within a vendor and switching vendors ends up being the same task. So you quickly solidify on either

1) never upgrading, although with these being cloud services that's not necessarily feasible, and since LLMs are far from a local maxima in quality that'd quickly leave your stack obsolete

or

2) being forced to be robust, which makes it easy to migrate to other vendors

Re:It's OpenAI's world, we're just living in it by iamflimflam1 on Friday October 10, 2025 @02:25PM (#45542146)
If you are doing things “properly” then you have good evals that let you test the behaviour of different LLMs and see if they work for your problem.

The vendors have all standardised on OpenAIs API surface - you can use OpenAIs SDK with a number of providers - so switching is very easy. There are also quite a few services that offer this as a service.

The real test is does a different LLM work - hence the need to evals to check.

Re:It's OpenAI's world, we're just living in it by jgalt212 on Friday October 10, 2025 @04:21PM (#45543345)
If you took any of current top 3 models from me, I would not miss the deleted one in the least. I run almost every non-trivial prompt through multiple models anyway.
Re:It's OpenAI's world, we're just living in it by BeetleB on Friday October 10, 2025 @04:28PM (#45543428)
> Each vendor's offering has its own peculiar prompt quirks, does it not?

That applies even when you switch models within a vendor, though.

Re:It's OpenAI's world, we're just living in it by impossiblefork on Friday October 10, 2025 @05:34PM (#45544028)
It's reasonable to question it, but there's a fun Chinese paper (https://arxiv.org/abs/2507.15855) where they attempt to create a system of prompts that can be used with commercial LLMs to solve hard maths problems.

It turns out that they can use the same prompt system for all of them, with no changes and still solve 5/6 IMO problems. I think this is possibly iffy, since people might have updated the models etc., but it's pretty obvious that this kind of thing is how OpenAI are doing their multi-stage thinking thing for maths internally.

Consequently if prompt systems are this transferable for these hard problems, why wouldn't both they and individual prompts, be highly transferable in general?

Re:It's OpenAI's world, we're just living in it by psychoslave on Friday October 10, 2025 @07:08PM (#45544852)
No, changing rdbms is a totally different challenge. They provide predictible reproducible output which are very sensitive and brittle to the slightest minor change in the input, while with LLM you expect the exact opposite.

Changing the LLM backend in some IDE is as complicated as selecting an option in a dropbox for those who integrate such a feature. They are other scenarios where it might be a bit more complicated to transition of course, but that's it.