Adityav369
Re:Don't bother parsing: Just use images for RAG
on Monday July 21, 2025 @06:25PM (#44641092)
You can ask the model to describe the image, but that is inherently lossy. What if it is a chart and the model gets most x, y pairs, but the user asks about a missing "x" or "y" value. Presenting the image at inference is effective since you're guaranteeing that the LLM is able to answer exactly the user's question. The only blocker here becomes how good retrieval is, and that's a smaller problem to solve. This approach allows us to only solve for passing in relevant context, the rest is taken care of by the LLM, otherwise the problem space expands to correct OCR, parsing, and getting all possible descriptions to images from the model.
Re:Don't bother parsing: Just use images for RAG
on Monday July 21, 2025 @03:51PM (#44639688)
Yeah the fine tuning is definitely the best part.
Often, the blocker becomes high quality eval sets (which I guess always is the blocker).
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Friday April 25, 2025 @01:27AM (#43790392)
Mainly to maintain layout information. Also search becomes easier this way.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @08:08PM (#43767486)
We do use ColQwen! Currently 2, but upgrading to 2.5 soon :)
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @06:39PM (#43766962)
Yeah we had an overload on the ingestion queue. If you try again will be much faster as we just moved to a beefier machine. (The previous ingestion will still work since it is in queue, but new ones will be faster)
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @06:36PM (#43766945)
Depending on the use case, it happily runs on my MacBook air M2 16GB ram with mps for small pdfs, and searching over 100-150 documents with colpali takes a 2-ish minutes. Very rough numbers. For ingestion, takes around 15-20-ish seconds a page, which is on the slower end. On an A100, it takes 4-5 seconds per page for ingestion using Colpali to run (we haven't performance optimized, or optimized batch sizes yet tho). Without Colpali it is much faster. Ingestion doesn't change much as size grows.
I'd be happy to report back after some testing, we are looking to optimize more of this soon, as speed is somewhat of a missing piece at the moment.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @05:46PM (#43766631)
For ingesting graphs, you can define a filter, or certain document ids. When updating, we look at if any other docs are added with that filer (or you can specify new doc ids). We then do entity and relationship extraction again, and do entity resolution with the existing graph to merge the two.
Creating graphs and entity resolution are both tunable with overrides, you can specify domain specific prompts and overrides (will add a pharma example!) (https://docs.morphik.ai/python-sdk/create_graph#parameters). I tried to add code, but was formatting badly, sorry for the redirect.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @05:41PM (#43766596)
You can run this fully locally using Ollama for inference, although you'll need larger models and a beefy machine for great results. On my end llama 3.2 8B does a good job on technical docs, but bigger the better lol.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @05:37PM (#43766567)
Thanks we should have been more clear. The part in ee is our UI, which can be used to test or in dev environments. The main code, including API, SDK, and the entire backend logic is MIT expat.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @05:32PM (#43766524)
Do you mean ingesting the extracted rectangles/ bounding boxes? We're actually working on bounding boxes, this is a good insight and we can add this to the product. However, the way we ingest is literally converting each page to an image then embedding that so the text, layout, diagrams are all encoded in. Would like to know what the exact use case is, can help you better
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @05:16PM (#43766392)
Depends on your document types.
If you're using txts, then plain RAG built on top of any vector database can suffice depending on your queries (if they directly reference the text, or can be made to, then similarity search is good enough). If they are cross document, setting a high number of chunks with plain RAG to retrieve might also do a good job.
If you have tables, images, etc. then using a better extraction mechanism (maybe unstructured, or other document processors) and then creating the embeddings can also work well.
I'd say if docs are simple, then just building your own pipeline on top of a vector db is good!
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @02:58PM (#43765182)
We have two ingestion pathways: 1. regular OCR + text embeddings; 2. Colpali. We've observed that Colpali does a much better job with tables since it can encode positional stuff and layouts as well.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @02:33PM (#43764961)
The paid part applies to the ui-component which provides a chat user interface. The core code, SDK, api is all under MIT license.
Re:Show HN: Morphik – Open-source RAG that understands PDF images, run...
on Tuesday April 22, 2025 @01:41PM (#43764557)
Thanks for pointing that out! Fixed it.
For the metadata extraction, we save these as Column(JSONB) for each documents which allows it to be changed on the fly.
Although, I keep wondering if it would have been better to use something like mongodb for this part, just because it's more natural.
Please let me know if you have questions and how it works out for you.
Show HN: Morphik – Open-source RAG that understands PDF images, runs locally
https://github.com/morphik-org/morphik-core
Hey HN, we’re Adi and Arnav. A few months ago, we hit a wall trying to get LLMs to answer questions over research papers and instruction manuals. Everything worked fine, until the answer lived inside an image or diagram embedded in the PDF. Even GPT‑4o flubbed it (we recently tried O3 with the same, and surprisingly it flubbed it too). Naive RAG pipelines just pulled in some text chunks and...
Re:Show HN: Morphik – Open-source MCP server for technical document se...
on Sunday April 06, 2025 @04:39PM (#43604786)
Apologies for the poor formatting!
Features:
- Multimodal search across text, diagrams, and videos
- Natural language knowledge base management
- Fully open-source with responsive support
Show HN: Morphik – Open-source MCP server for technical document search
https://docs.morphik.ai/using-morphik/mcp
Hey HN - we built Morphik MCP to solve a common problem: finding specific information across scattered technical docs. We've experimented with GraphRAG, ColPali, contextual embeddings, and more. MCP emerged as the solution that unifies these approaches.Features: - Multimodal search across text, diagrams, and videos - Natural language knowledge base management - Fully open-source with responsive...
Show HN: I built an open-source vision RAG that embeds images and text together
https://docs.morphik.ai/concepts/colpali
My brother and I have been working on Morphik: an open-source multimodal database. (Github: https://github.com/morphik-org/morphik-core)After experimenting with various AI models, we realized that they were particularly bad at answering questions which required retrieving over images and other multimodal data.That is, if I uploaded a 10-20 page PDF to ChatGPT, and ask it to get me a result from...
Show HN: DataBridge: Open-Source Multi-Modal Parser and Retriever for LLMs
https://github.com/databridge-org/databridge-core
Hey HN!When testing complex PDFs (with diagrams, formulas, etc.), I found Gemini would straight up hallucinate, while GPT-4o asked me to run a script to parse the file: "It looks like you uploaded a PDF, but I can't directly analyze it unless I extract its contents. Do you want me to process the document and find the relevant data related to IRR (Internal Rate of Return)?"—but that script only...
Show HN: DataBridge: Rule-Based Metadata Extraction, PII Redaction, and More
Show HN: Build video AI agents with DataBridge
https://www.youtube.com/watch?v=1gT8PXzkGHg
DataBridge is a multimodal database for building multi agent AI systems. It's open source and can run fully locally (with easy model swaps to run your favorite models).Github: https://github.com/databridge-org/databridge-core Docs: https://databridge.gitbook.io/databridge-docs
Re:Show HN: I built an offline open-source RAG system DataBridge
on Sunday January 19, 2025 @11:48AM (#42758672)
Good question! You can use any open source or private model you want with this (just by changing one line in `databridge.toml`), whereas notebook lm is limited to gemini. While our UI component allows it to be used as an open source notebook lm alternative, it is developed for developers building AI apps and doing context management. We are working on adding techniques like CAG (cache augmented generation) and graphRAG for better and more flexible querying, and have custom embedding pipelines for different data types.
Show HN: I built an offline open-source RAG system DataBridge
https://github.com/databridge-org/databridge-core
Hey HN!Over the past few weeks, I’ve been working on DataBridge, an open-source solution for data ingestion and querying across text, PDFs, images, and videos.In our latest update, we’ve added a fully local deployment option:- No internet required – Runs entirely offline. - Customizable Models – Supports any LLM and embedding model via Ollama (with options for any other private providers) -...
Show HN: [Open-source Project] DataBridge (Modular multi-modal RAG solution)
https://github.com/databridge-org/databridge-core
For the past few weeks, I've been working on DataBridge, an open-source solution for easy data ingestion and querying. We support text, PDFs, images, and as of recently, we've added a video parser that can analyze and work well over frames and audio. We are also adding object tracking to improve video ingestion and context, and plan to do this for various data types.To get started, you can find...
Re:Show HN: Automate your social media workflow
on Sunday August 11, 2024 @02:48PM (#41218445)
Thank you. Definitely! We have cross posting for linkedin and instagram as of now. Integrating ActivityPub/ Fediverse is on our list.
Have you personally faced this issue/ know of anyone who has been looking for a solution to fill this market hole? Would love to know more.
Re:Show HN: Automate your social media workflow
on Sunday August 11, 2024 @02:40PM (#41218388)
How to use/demo video: https://www.youtube.com/watch?v=Q6twwYsYNKI&t=1s
Hey HN! My brother and I are building BrandSync AI to address a problem we identified while talking with businesses: Businesses have existing brand assets like catalogs, PDFs, images, etc, but struggle to translate these into effective social media content. BrandSync AI leverages these assets, along with platform-specific best practices, to generate social media posts.
We'd love to get your feedback on the product and would love to connect if this can help solve any of y'alls problems. Let us know what you think!
Show HN: Automate your social media workflow
BrandSync AI is a platform that automates your social media workflow. Bring in any documents: (catalogs, code, technical documents, etc.) and let BrandSync AI create platform optimized social media posts for you in seconds.