News for hackers. Stuff that matters.

A comment on Anthropic - Introducing 100K Token Context Windows, Around 75,000 Words. Parent comment

Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by summarity on Thursday May 11, 2023 @03:58PM (#35907625)
That is solved by hypothetical embeddings.

Background: https://summarity.com/hyde

Demo: https://youtu.be/elNrRU12xRc?t=1550 (or try it on findsight.ai and compare results of the "answer" vs the "state" filter)

For even deeper retrieval consider late interaction models such as ColBERT

[ Parent | on HN ]
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by akiselev on Thursday May 11, 2023 @04:15PM (#35907803)
Any material comparing the different embedding models? I'm working on information retrieval from government documents and without any ML experience it's daunting
[ Parent | on HN ]
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by adamgordonbell on Saturday May 13, 2023 @07:45AM (#35927458)
I'm not understanding how that works compared to having the full text?

Does the embedding structure somehow expose the themes? And if so, is it more the embeddings that are answering the question by how it groups things?

[ Parent | on HN ]