Re:The Pile is a 825 GiB diverse, open source language model...
by doctorpangloss on Thursday March 07, 2024 @02:27PM
(#39633531)
It’s enough for pre training to later tackle specific NLP tasks.
To get something interesting you would have to generate an instruct dataset from it. It would have to cover a diverse range of tasks. The completions themselves do not make LLMs manifest knowledge and reasoning, a large and diverse instruct dataset does.
To get something interesting you would have to generate an instruct dataset from it. It would have to cover a diverse range of tasks. The completions themselves do not make LLMs manifest knowledge and reasoning, a large and diverse instruct dataset does.