Re:InternLM2
by pilotneko on Monday April 01, 2024 @09:17AM
(#39893864)
I experimented with this model and vLLM around a month ago. The long context length is attractive, but it was incredibly slow on a g5.12xlarge (4 NVIDIA A10G GPUs). I actually could not get it to respond for single examples longer than 50K tokens.
[ Parent
| on HN ]
There are no comments on this comment yet.