We also use a slightly different strategy to build a lossless memory system without any Vector DBs: https://github.com/VectifyAI/ChatIndex
Hope to have more discussions around this topic!
OCR tools often lose document structure or output wrong heading levels, since they only see one page at a time — losing context, structure, and continuity across pages.PageIndex OCR is a long-context OCR model designed to preserve the global structure of documents. It recognizes true hierarchy and semantic relationships across document pages, aiming to address issues common in traditional OCR....
Would love feedback and suggestions.
Current vector-based RAG systems rely on semantic similarity to retrieve content — but similarity ≠ relevance.In domains like finance or law, the answer isn't just in the paragraphs that look similar to the query — it's in the sections that human experts would look at first. Embedding models don't know that, and fine-tuning them to encode this domain logic is expensive and inflexible.We built...
Would love feedback and suggestions!
Current embedding-based RAG systems primarily rely on semantic similarity. Given a document and a query, the system usually retrieves multiple sections that appear semantically relevant. However, in domain-specific applications, such as financial analysis or legal research, users often have domain-specific preferences for which parts of a document to consult first. These preferences are...
https://github.com/VectifyAI/PageIndex
We built PageIndex, a document indexing system that turns documents into hierarchical search trees to support reasoning-based RAG.Traditional vector-based RAG often struggles with retrieval accuracy because it optimizes for similarity, not relevance. But what we really need in retrieval is relevance — which requires reasoning. When working with professional documents that demand domain...
Open to feedback and suggestions.
Traditional vector-based RAG often struggles with retrieval accuracy because it optimizes for similarity, not relevance. But what we truly need in retrieval is relevance, which requires reasoning. When working with professional documents that require domain expertise and multi-step reasoning, vector-based RAG and similarity search often fall short.So we started exploring a more reasoning-driven...
We were frustrated by vector-based RAG systems that rely on semantic similarity and often fail on long, domain-specific documents. In these contexts, domain-specific terminology tends to be semantically similar, making it hard to retrieve the exact content users need. It’s also difficult to incorporate expert knowledge or user preferences effectively. So we started exploring a more...