News for hackers. Stuff that matters.
Re:RWKV Language Model by lostmsu on Thursday January 02, 2025 @04:48AM (#42573103)
Any LLM is "streaming capable".
Re:RWKV Language Model by Xmd5a on Thursday January 02, 2025 @08:45AM (#42574340)
https://github.com/mit-han-lab/streaming-llm

On a side node, and that's what led me to the link above, I wonder if it would be possible to chain N streaming LLMs in an agent workflow and get a final output stream almost instantaneously without waiting for N-1 LLM to complete their reply.