Falcon 40B LLM (which beats Llama) now Apache 2.0

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by Zemtomo on Friday June 02, 2023 @09:17AM (#36164646)

This is bleeding edge stuff.

All advances in this direction do indicate that it will be easier and easier for more people to do things with it.

This doesn't need to work for everyone.

A 4090 costs today 2k, the 3090 with also 24gb costs today 1k and costed 2k.

[ Parent | on HN ]

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by caeril on Friday June 02, 2023 @09:44AM (#36164955)

Any particular reason not to run this model on a single Jetson AGX Orin 64GB?

GP Core count is much lower than than the 4090 but it still does 275 int8 TOPS for only $2k

[ Parent | on HN ]

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by Zemtomo on Friday June 02, 2023 @11:31AM (#36166499)

I'm the wrong person to ask but performance wise a 4090 has over a pflop and a Google search also showed a factor of 3-4 for a 3090.

[ Parent | on HN ]

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by cjbprime on Friday June 02, 2023 @09:43PM (#36172897)

The relevant performance metric here is memory amount and bandwidth.

[ Parent | on HN ]