Re:The tide is shifting: 1.3B outperforms 7B Llama 2
by brucethemoose2 on Tuesday September 12, 2023 @06:56PM
(#37489750)
> How do you curate a general purpose model's dataset without throwing the baby out with the bathwater
Time, money, and human work. The pruning process needs focused input from experts other than ML researchers and data scientists.
Re:The tide is shifting: 1.3B outperforms 7B Llama 2
by mycall on Tuesday September 12, 2023 @10:33PM
(#37491520)
Maybe the Cyc team could provide some generalization tips.
Re:The tide is shifting: 1.3B outperforms 7B Llama 2
by waste_monk on Wednesday September 13, 2023 @03:20AM
(#37493290)
>How do you curate a general purpose model's dataset without throwing the baby out with the bathwater ?
Maybe we could train an AI to do it ;)
This is true and most researchers understand this on some level (https://arxiv.org/abs/2305.07759) but understanding it doesn't really make the problem any easier. How do you curate a general purpose model's dataset without throwing the baby out with the bathwater ?