Mistral: Our first AI endpoints are available in early access

Re:Mistral: Our first AI endpoints are available in early ac... by stavros on Monday December 11, 2023 @08:26AM (#38600425)

Really? Well that's very exciting. I don't care about wasting my battery if it can do my menial tasks for me, battery is a currency I'd gladly pay for this use case.

[ Parent | on HN ]

Re:Mistral: Our first AI endpoints are available in early ac... by TeMPOraL on Monday December 11, 2023 @10:06AM (#38601260)

Maybe this way, we'll not just get user-replaceable batteries in smartphones back - maybe we'll get hot-swappable batteries for phones, as everyone will be happy to carry a bag of extra batteries if it means using advanced AI capabilities for the whole day, instead of 15 minutes.

[ Parent | on HN ]

Re:Mistral: Our first AI endpoints are available in early ac... by stavros on Monday December 11, 2023 @10:14AM (#38601326)

Or maybe we'll finally get better batteries!

Though I guess that's not for lack of trying.

[ Parent | on HN ]

Re:Mistral: Our first AI endpoints are available in early ac... by brandall10 on Monday December 11, 2023 @01:29PM (#38603452)

I think we're at least a couple generations away where this is feasible for these models, unless say it's for performing a limited number of background tasks at fairly slow inference speed. SOC power draw limits will probably limit inference speed to about 2-5 tok/sec (lower end for Mixtral which has the processing requirements of a 14B) and would suck an iPhone Max dry in about an hour.

[ Parent | on HN ]