News for hackers. Stuff that matters.
Re:Phi-3 Technical Report by irjustin on Tuesday April 23, 2024 @12:36AM (#40128552)
I'm pretty naive so please forgive it's a stupid question.

To me, what the parent comment is saying is that even though the benchmarks are cool, it's not super helpful to the every day person. Because if you can't chat with it very well (even for a narrow context) what utility does it have with great benchmarks?

Re:Phi-3 Technical Report by svnt on Tuesday April 23, 2024 @01:08AM (#40128703)
Both are saying the same thing: in order for the base model that is phi to perform well as a chat agent, it would need to be tuned for that purpose before its benchmark results could have real-world value.
Re:Phi-3 Technical Report by imjonse on Tuesday April 23, 2024 @01:49AM (#40128906)
From this report. Phi-2 was not instruct tuned indeed.

"Our models went through post-training with both supervised instruction fine-tuning, and preference tuning with DPO. We have worked on generating and curating various instruction and preference data. This has improved the model chat capabilities, robustness, as well as its safety."