News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s. Parent comment

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by WanderPanda on Monday February 19, 2024 @01:52PM (#39433272)
What do you mean by non-deterministic hardware? cuBLAS on a laptop GPU was deterministic when I tried it last iirc
[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @02:05PM (#39433406)
Non-deterministic timing characteristics.
[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by frozenport on Monday February 19, 2024 @02:07PM (#39433434)
Tip of the ice-berg.

DRAM needs to be refreshed every X cycles.

This means you don't know the time it takes to read from memory. You could be reading at a refresh cycle. This circuitry also adds latency.

[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by LtdJorge on Monday February 19, 2024 @07:17PM (#39436488)
OP says SRAM, which doesn't decay so no refreshing.
[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by ndjdbdjdbev on Tuesday February 20, 2024 @02:35AM (#39438862)
Timing can simply mean the FETs that make up the logic circuits of a chip. The transition from high to low and low to high has a minimum safe time to register properly...
[ Parent | on HN ]