News for hackers. Stuff that matters.

A comment on Cerebras-GPT vs. LLaMA AI Model Performance Comparison. Parent comment

Re:Cerebras-GPT vs. LLaMA AI Model Performance Comparison by sebzim4500 on Wednesday March 29, 2023 @07:40PM (#35365544)
I think you are likely wrong, but given neither of us are going to spend millions of dollars training two versions of GPT-3 we will have to agree to disagree. Meta seems to agree with me, since when they trained LLaMA they used one token per digit.
[ Parent | on HN ]
Re:Cerebras-GPT vs. LLaMA AI Model Performance Comparison by sillysaurusx on Wednesday March 29, 2023 @07:51PM (#35365681)
> neither of us are going to spend millions of dollars training two versions of GPT-3 we will have to agree to disagree

I encourage you to apply to TRC https://sites.research.google/trc/about/

You'll be able to access millions of dollars worth of compute. It's how I got my start.

I love when I'm mistaken, since that's how science is pushed forward – we can't be certain we're right, only that we're not wrong yet. So it would be delightful if you formulate this into a testable hypothesis and falsify it yourself.

You can use The Pile to train your GPT: https://pile.eleuther.ai/

[ Parent | on HN ]