News for hackers. Stuff that matters.

A comment on Ask HN: Which LLMs can run locally on most consumer computers. Parent comment

Re:Ask HN: Which LLMs can run locally on most consumer compu... by artemisart on Wednesday May 22, 2024 @05:37PM (#40446933)
No, the Nvidia 4070 Ti has much higher performance, TOPS is for integer operations, the 4070 Ti has ~40 float32 TFLOPS and 641 TOPS https://www.nvidia.com/fr-fr/geforce/graphics-cards/40-serie... (which I would say would be peak TOPS for int4 operations, comparing it to the 4080 datasheet, and a bit more than half that for int8 operations) https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvid... page 34. I did not find the datasheet for 4070 Ti.
[ Parent | on HN ]

There are no comments on this comment yet.