10M Tokens LLM Context

Posted by nsky-world on Friday February 02, 2024 @12:52PM
from the github.com dept.

( 1 comment | on HN )

Re:10M Tokens LLM Context by nsky-world on Friday February 02, 2024 @12:52PM (#39231783)

KVQuant: Towards Enabling 10 Million Context Length For LLM Inference through KV Cache Quantization

[ Parent | on HN ]