Re:Non-determinism in GPT-4 is caused by Sparse MoE
by visarga on Saturday August 05, 2023 @12:32AM
(#37009045)
Only on specific domains, these models don't become generalists like GPT-4, they can become task experts for a single task.
[ Parent
| on HN ]
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by og_kalu on Saturday August 05, 2023 @08:13AM
(#37011246)
Fine-tuned MedPalm is worse than GPT-4 on most Medical Challenge Tests.
Fine-tuned Minerva is much worse on arithmetic benchmarks.
The LLM space is just different. There's no guarantee a fine-tuned model will beat a bigger generalist one.