Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by great_psy on Wednesday January 17, 2024 @02:52PM
(#39032557)
This a pretty strong claim with zero data to back it up
[ Parent
| on HN ]
Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by lxe on Wednesday January 17, 2024 @04:07PM
(#39033453)
While I lack specific data, my intuition is based on observed trends in AI model development. I believe some other models that claimed such numbers excelled in benchmarks but fell short in real-world applications. Further research can validate this claim, and I welcome a balanced discussion.
Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by travisporter on Wednesday January 17, 2024 @04:10PM
(#39033494)
It does seem incredible that chatgpt has so much expertise in literally everything. Does this mean you can beat chatgpt by creating smaller "experts" and directing questions to each?
Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by great_psy on Thursday January 18, 2024 @04:10PM
(#39047787)
See mixture of experts. It’s likely what chatGPT does in the backend.
Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by eightysixfour on Wednesday January 17, 2024 @04:03PM
(#39033409)
Every small model that has outperformed GPT-4 has proven to be an overfit, so I would say it is the obvious claim, and any claim opposite that is what we should be skeptical of.
Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by anon373839 on Thursday January 18, 2024 @01:45AM
(#39038744)
With the exception of task specialization. Fine-tuning a small model such as Mistral 7B on a specific set of tasks can outperform using GPT-4 on those tasks, and with cheaper and faster inference.
Re:Wavecoder – a CodeLLM with 6.7B params scoring just behin...
by eightysixfour on Thursday January 18, 2024 @11:16AM
(#39043587)
Not on the leaderboards mentioned here. That’s my point, you can overfit for specific tasks, you can’t beat them on multi-task leaderboards without training on the test data.