Re:Jina AI Launches First Open-Source 8K Text Embedding, Riv...
by woadwarrior01 on Thursday October 26, 2023 @06:23AM
(#38023780)
BERT style encoder-only models, like the embedding model being discussed here, don't need a KV cache for inference. A KV cache is only needed for efficient inference with encoder-decoder and decoder-only (aka GPT) models.
[ Parent
| on HN ]
There are no comments on this comment yet.