Re:Mistral: Our first AI endpoints are available in early ac...
by brandall10 on Monday December 11, 2023 @01:29PM
(#38603452)
I think we're at least a couple generations away where this is feasible for these models, unless say it's for performing a limited number of background tasks at fairly slow inference speed. SOC power draw limits will probably limit inference speed to about 2-5 tok/sec (lower end for Mixtral which has the processing requirements of a 14B) and would suck an iPhone Max dry in about an hour.
[ Parent
| on HN ]
There are no comments on this comment yet.