Groq runs Mixtral 8x7B-32k with 500 T/s

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by frozenport on Monday February 19, 2024 @02:07PM (#39433434)

Tip of the ice-berg.

DRAM needs to be refreshed every X cycles.

This means you don't know the time it takes to read from memory. You could be reading at a refresh cycle. This circuitry also adds latency.

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by LtdJorge on Monday February 19, 2024 @07:17PM (#39436488)

OP says SRAM, which doesn't decay so no refreshing.

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by ndjdbdjdbev on Tuesday February 20, 2024 @02:35AM (#39438862)

Timing can simply mean the FETs that make up the logic circuits of a chip. The transition from high to low and low to high has a minimum safe time to register properly...

[ Parent | on HN ]