vLLM: 24x faster LLM serving than HuggingFace Transformers

Re:vLLM: 24x faster LLM serving than HuggingFace Transformers by wskwon on Tuesday June 20, 2023 @08:13PM (#36412592)

Thanks for the explanation! I believe the two ideas are basically orthogonal. FlashAttention reduces memory read/writes, while PagedAttention reduces memory waste.

[ Parent | on HN ]

There are no comments on this comment yet.