Show HN: Morphik – Open-source RAG that understands PDF images, runs locally

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by thot_experiment on Tuesday April 22, 2025 @05:33PM (#43766537)

I'd love to have something like this but calling a cloud is a no-go for me. I have a half baked tool that a friend of mine and I applied to the Mozilla Builders Grant with (didn't get in), it's janky and I don't have time to work on it right now but it does the thing. I also find myself using OpenWebUI's context RAG stuff sometimes but I'd really like to have a way to dump all of my private documents into a DB and have search/RAG work against them locally, preferably in a way that's agnostic of the LLM backend.

Does such a project exist?

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by Adityav369 on Tuesday April 22, 2025 @05:41PM (#43766596)

You can run this fully locally using Ollama for inference, although you'll need larger models and a beefy machine for great results. On my end llama 3.2 8B does a good job on technical docs, but bigger the better lol.

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by thot_experiment on Tuesday April 22, 2025 @07:48PM (#43767366)

Ahh, I didn't see that, I just saw them talking about a free tier or whatever and my eyes glazed over. I'll try it out with Mistral-small 3.1 at some point tonight, I've been having really great results with it's multimodal understanding.

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by mrtimo on Tuesday April 22, 2025 @10:44PM (#43768211)

how would you use this within open-web-ui locally?

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by osigurdson on Tuesday April 22, 2025 @08:41PM (#43767627)

Just curious, are you fine with running things in your own AWS / Azure / GCP account or do you really mean that the solution has to be fully on-premise?

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by thot_experiment on Wednesday April 23, 2025 @12:30AM (#43768616)

Airgapped. It really makes threat modelling so so soooo much easier. It's temporal so if I were being attacked by a state level actor exfiltration is possible but this specific application I either have the data live and no internet, or internet and no data. I also have some lesser stuff that I allow on-prem w/ internet and just trust the firewall, but absolutely no way am I doing any sensitive data storage or inference in the cloud.

Since people will be curious, one lesser thing I used this for is a diary/assistant and it's nice to have the peace of mind that I can dump my inner most thoughts without any concern for oversharing.

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by rank0 on Wednesday April 23, 2025 @01:06AM (#43768773)

What kind of hardware do you need for this setup?

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by thot_experiment on Wednesday April 23, 2025 @02:24AM (#43769204)

A computer with a couple gaming GPUs, a lan cable you can unplug and an encrypted external hard drive to offline your sensitive data.

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by ArnavAgrawal03 on Wednesday April 23, 2025 @01:35PM (#43774589)

totally agree that air-gapped provides unparalleled peace of mind. That's a major reason why we have strong support for local deployment. Nice to know that our hypothesis is somewhat accurate :)

[ Parent | on HN ]

Re:Show HN: Morphik – Open-source RAG that understands PDF i... by oceansweep on Wednesday April 23, 2025 @07:07PM (#43777647)

Hey yes, I’m building exactly that.

https://github.com/rmusser01/tldw

I first built a POC in gradio and am now rebuilding it as a FastAPI app. The media processing endpoints work but I’m still tweaking media ingestion to allow for syncing to clients(idea is to allow for client-first design). The GitHub doesn’t show any of the recent changes, but if you check back in 2-3 weeks, I think I’ll have the API version pushed to the main branch.

[ Parent | on HN ]