News for hackers. Stuff that matters.

A comment on Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20. Parent comment

Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by sabareesh on Tuesday May 28, 2024 @04:43PM (#40505413)
Well here is a comment on 4090 https://github.com/karpathy/llm.c/discussions/481#discussion...
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by huac on Wednesday May 29, 2024 @09:54AM (#40512047)
25% MFU :( maybe because of the P2P nerf?
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by sabareesh on Wednesday May 29, 2024 @12:13PM (#40513555)
This much bigger model (500M), P2P is enabled via Mailbox. It is expected because of memory to compute ratio
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by huac on Wednesday May 29, 2024 @12:20PM (#40513671)
can you elaborate?
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by anthonix1 on Wednesday May 29, 2024 @10:22AM (#40512312)
Maybe get a 7900 XTX. 122 TFLOPS of BF16/FP16 for less than $1k and I'm getting 55.4% MFU
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by sabareesh on Wednesday May 29, 2024 @12:13PM (#40513564)
These are not apples to apple comparison, as this is running across GPU and much bigger model
[ Parent | on HN ]