Tenstorrent Unveils High-End Wormhole AI Processors, Featuring RISC-V

Re:Tenstorrent Unveils High-End Wormhole AI Processors, Feat... by imtringued on Sunday July 21, 2024 @06:36AM (#41024013)

I agree. Their 70B parameter benchmarks are better than 7B. Something is wrong with their software and they need to fix it. They get 4000 token/s on Falcon 7B, which is what you should expect from processing a batch of prompts. The per user performance is atrocious and their hardware should be at least twice as fast as a moderately overclocked DDR5 system.

[ Parent | on HN ]

Re:Tenstorrent Unveils High-End Wormhole AI Processors, Feat... by qeternity on Sunday July 21, 2024 @07:39AM (#41024365)

You can do 4k t/s at large batch on last gen consumer gear. You’re right their 70B are more competitive but still very much off the mark. If these cards cost $500 instead of $1400 then maybe it would be more compelling.

[ Parent | on HN ]