News for hackers. Stuff that matters.
Re:LLM based agents as Dungeon Masters by someone3876 on Tuesday January 14, 2025 @03:43PM (#42703570)
Have you actually used Gemini? I use it a lot for translation, and its context window is more like 150k tokens, rather than the 2M context window they say it has.
Re:LLM based agents as Dungeon Masters by Der_Einzige on Tuesday January 14, 2025 @03:57PM (#42703738)
Be that as it may, long context window models which are good are not a mirage. By say late 2027, when the LLM providers figure out that they're using the wrong samplers, they will figure out how to get you 2 million output tokens per LLM call which stay coherent.
Re:LLM based agents as Dungeon Masters by gwern on Monday January 20, 2025 @07:50PM (#42775039)
My apologies to all the D&D sessions which take more than 150k tokens, then.