I honestly don't understand this latency obsession for LLMs. You are loading millions of parameters sequentially for each matrix. The access pattern is perfectly predictable. I just ran llama.cpp in with profiling and 99.9% of the time is spent in matrix multiplication. This shocked me, honestly, because I genuinely thought that there is going to be much more variety.