Re:Microsoft Phi-2 model changes licence to MIT
by Y_Y on Saturday January 06, 2024 @06:43AM
(#38890622)
The Elo scoring system is named after a Hungarian man called Arpad Elo (a simplification of his original more Hungarian name). My phone helpfully miscorrects it to "ELO" probably because it prefers Jeff Lynne's Electric Light Orchestra. Anyway,
Elo is a proper name, not an acronym!
Re:Microsoft Phi-2 model changes licence to MIT
by moffkalast on Saturday January 06, 2024 @07:10AM
(#38890788)
TIL, interesting. I always figured it must be some kind of abbreviation.
Re:Microsoft Phi-2 model changes licence to MIT
by stavros on Saturday January 06, 2024 @05:22AM
(#38890197)
That makes sense, thank you. There seems to be some inflation, as Mistral is supposed to be GPT-3.5-level, and Mixtral is supposed to be nearer GPT-4, but yeah, it sounds suspicious in practice, even though Mistral is very good.
Re:Microsoft Phi-2 model changes licence to MIT
by moffkalast on Saturday January 06, 2024 @05:38AM
(#38890270)
Well in some things it totally can be to some extent, yes. You can almost certainly get a Mistral 7B fine tuned for a specific thing (e.g. coding) and it will likely be about as good as 3.5 in that specific thing (not a super high bar in objective terms). For all the other areas it may suffer in performance relative to its original self, but for some applications that's fine. As for GPT-4 it's about 120 ELO points [0] above Mixtral, and that's even the distilled turbo version. Not even close imo, especially when Mixtral is far less censored.
Both 3.5 and 4 have changed drastically over the past year with continued fine tuning, quantization, etc. so what some people consider their level is not exactly a fixed point either.
[0] The actual leaderboard I'm referencing, it has its biases but it's the most generally indicative thing available right now: https://chat.lmsys.org
Re:Microsoft Phi-2 model changes licence to MIT
by regularfry on Saturday January 06, 2024 @06:03AM
(#38890387)
Mixtral is only 32 ELO ahead of the best 7B model on that leaderboard, although I suspect that might be understating the difference.
You can try Phi-2 with wasm here, I mostly just get gibberish out of it though: https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-...
Mixtral is the only model that properly matches 3.5 at the moment.