News for hackers. Stuff that matters.

A comment on Falcon 40B LLM (which beats Llama) now Apache 2.0. Parent comment

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by caeril on Friday June 02, 2023 @09:44AM (#36164955)
Any particular reason not to run this model on a single Jetson AGX Orin 64GB?

GP Core count is much lower than than the 4090 but it still does 275 int8 TOPS for only $2k

[ Parent | on HN ]
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by Zemtomo on Friday June 02, 2023 @11:31AM (#36166499)
I'm the wrong person to ask but performance wise a 4090 has over a pflop and a Google search also showed a factor of 3-4 for a 3090.
[ Parent | on HN ]
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by cjbprime on Friday June 02, 2023 @09:43PM (#36172897)
The relevant performance metric here is memory amount and bandwidth.
[ Parent | on HN ]