Llama3 implemented from scratch

Re:Llama3 implemented from scratch by andy99 on Sunday May 19, 2024 @03:58PM (#40409366)

I don't want to be dismissive, it's a fun project, but this has been done a lot already - maybe not with llama3 but the architecture is basically the same as llama2. Look at the big list of from scratch implementations on Karpathys llama2.c page.

Is there something particularly different about this one?

Edit - guess not?

[ Parent | on HN ]

Re:Llama3 implemented from scratch by fifilura on Sunday May 19, 2024 @04:17PM (#40409469)

I think they learned a lot doing this? And they tried hard explaining each step!

[ Parent | on HN ]

Re:Llama3 implemented from scratch by rvz on Sunday May 19, 2024 @04:19PM (#40409475)

Well given the fast pace of AI, it should not be a surprise that this is similar to llama2 and that we’re seeing the n + 1 toy implementations and likely has bugs or leaks in the background.

You might as well look at llama.cpp for a serious and production grade implementation to learn from. Otherwise, nothing to see here.

> Is there something particularly different about this one?

Other than the immature lowercase, anime BS, etc, then…

No.

[ Parent | on HN ]

Re:Llama3 implemented from scratch by tildef on Sunday May 19, 2024 @04:40PM (#40409593)

There's literally an image of Anya pointing at Karpathy on this GitHub page.

[ Parent | on HN ]

Re:Llama3 implemented from scratch by _giorgio_ on Monday May 20, 2024 @06:36AM (#40414069)

What are your favourite implementations of a GPT? I like a lot the video series by Karpathy.

Anyway, I'll take a look at this too, not sure if it has inference and training. Having just inference would be a disappointment.

[ Parent | on HN ]