News for hackers. Stuff that matters.
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by bookaway on Monday February 26, 2024 @10:54AM (#39512683)
It's also a good play to try to take resources away from local, self-hosted "Feasible AI" solutions. With compute resources, I think Microsoft hopes Mistral skews their focus and resources towards large models that can run only run in the cloud, trying to lure them away with the bait: "Don't you want to build the best AI possible, independent of compute?"

I'd be surprised if they didn't consider the notion that they are hitting to birds with one stone: OpenAI and Indie AI.

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by dhruvdh on Monday February 26, 2024 @11:27AM (#39513199)
It's not like Microsoft is working on "Windows AI Studio" [1], or released Orca, or Phi. It's not like there's any talk of AI PCs with mandatory TOPs requirements for Windows 12. Big bad Microsoft coming for your local AI, beware.

[1] https://github.com/microsoft/windows-ai-studio

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by whimsicalism on Monday February 26, 2024 @11:47AM (#39513527)
You’re downvoted for the snarky tone I guess, but you’re absolutely right
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by peteradio on Monday February 26, 2024 @12:15PM (#39513941)
Even easier to rug pull your own teams project than someone else's.
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by drakenot on Monday February 26, 2024 @10:10PM (#39519714)
The whole embrace, extend.. ?

> Mistral Remove "Committing to open models" from their website

That was 5 hours ago.

Without having insider details it is hard to know why, but the coincidence of timing with the Microsoft deal is not lost on me. It could have even been a stipulation.

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by Aerbil313 on Tuesday February 27, 2024 @03:39PM (#39529565)
I have no explanation for why Microsoft has started aggressively innovating again (with the introduction of Satya) than my theory that US DoD realized the country's tool of dominance in the future will be predominantly with tech superiority instead of military power. Microsoft's new strategy of running everything on the cloud aligns with this, even if it may have been also motivated by the fact that most people now only own a battery-constrained mobile device and laptops getting smaller and thinner.
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by smoldesu on Monday February 26, 2024 @11:43AM (#39513454)
I don't see why Mistral would acquiesce. Like the other comment says, Microsoft has a lot of chips on the table for local AI. They didn't even mention DirectML, ONNX or Microsoft's other local AI frameworks - suffice to say Microsoft does care about on-device AI.

So... would Mistral deliberately sabotage their low-end models to appease Microsoft's cloud demand? I don't think so. Microsoft probably knows that letting Mistral fall behind would devalue their investment. It makes more sense to bolster the small models to increase demand for the larger ones, at least from where I'm standing.

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by moneywoes on Tuesday February 27, 2024 @10:32AM (#39525239)
what’s the most promising?
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by smoldesu on Tuesday February 27, 2024 @03:31PM (#39529452)
If you're asking about Microsoft's APIs - I'd keep an eye on ONNX. It's the most ambitious, but also supports an insane amount of acceleration targets. It would be the proverbial "big guns" if vendors continued investing in more insular frameworks like Metal and CUDA.
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by foobiekr on Thursday February 29, 2024 @10:53AM (#39550992)
Money. How do they get revenue in your model?
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by teh_infallible on Monday February 26, 2024 @01:11PM (#39514783)
So.. Embrace, Extend, Extinguish?
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by isoprophlex on Monday February 26, 2024 @01:12PM (#39514798)
Moving straight from Embrace to Extinguish, why not!
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by loceng on Monday February 26, 2024 @01:23PM (#39514947)
From my understanding, which may be wrong, you only need the massive compute resources initially to create a compiled vector space LLM - and then that LLM once compiled can be run locally?

This is why anti-CSAM measures policy is possible so compiled-release LLMs can have certain vector spaces removed before release; but apparently people are creating cracks for these types of locks?

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by extr on Monday February 26, 2024 @06:11PM (#39517983)
You are a little confused. There’s no “compiling” of LLMs. It’s just once it’s trained, inference takes less compute than further training. So you can run things locally that you couldn’t necessarily train locally.

Not sure where you are getting the CSAM bit. We aren’t that good at blanking out weights in any kind of model, certainly not good enough to lobotomize specific types of content.

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by JyB on Monday February 26, 2024 @06:36PM (#39518219)
I love that you are using the word lobotomize.
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by loceng on Monday February 26, 2024 @06:54PM (#39518345)
Thanks for the clarification.

The CSAM bit seems to then be propaganda from at least one AI company putting out PR to falsely quell people's concerns about their LLMs being able to generate content involving children that's sexualized.

I've yet to see details of how much compute-minimum server requirements are necessary to run LLMs. Maybe you know a source who's compiling a list in a feature matrix that includes such details?

Re:Microsoft strikes deal with Mistral in push beyond OpenAI by dwaltrip on Monday February 26, 2024 @09:11PM (#39519273)
Large LLMs like gpt-3 and gpt-4 need very serious hardware. They have hundreds of billions of parameters (or more) which need to be loaded in memory all at once.
Re:Microsoft strikes deal with Mistral in push beyond OpenAI by Zuiii on Monday February 26, 2024 @11:21PM (#39520121)
Antitrust. Using their dominance in one market to destroy another. I hope the EU tares them a new one if the us doesn't.