News for hackers. Stuff that matters.
Re:Gemini 1.5 Pro by kiraaa on Thursday February 15, 2024 @06:40PM (#39390886)
maybe they are using ring attention, on top of their 128k model.
Re:Gemini 1.5 Pro by ein0p on Thursday February 15, 2024 @09:18PM (#39392313)
More likely some clever take on RAG. There’s no way that 1M context is all available at all times. More likely parts of it are retrievable on demand. Hence the retrieval-like use cases you see in the demos. The goal is to find a thing, not to find patterns at a distance
Re:Gemini 1.5 Pro by kiraaa on Friday February 16, 2024 @05:32AM (#39395295)
could be true, we can only speculate.