News for hackers. Stuff that matters.

A comment on Show HN: Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used). Parent comment

Re:Show HN: Pathway – Build Mission Critical ETL and RAG in ... by dxtrous on Friday June 14, 2024 @05:48PM (#40685456)
The main factor impacting the RAM requirement of the instance is the size of the data that you feed into it, especially if you need an in-memory index. (If you are curious about peak memory use etc., you can profile Pathway memory use in Grafana: https://github.com/pathwaycom/pathway/tree/main/examples/pro....)

One point to clarify is that "Pathway Community" is self-hosted, and the "8GB RAM - 4 cores" value is just a limit on the dimension of your own/cloud machine that the framework will effectively use. Currently, if you would like to get a "free" cloud machine to go with your project, we suggest going for "Pathway Scale" and reaching out through the #Developer Assist link - add a mention that you are interested in cloud credits. You can also go with 3rd party hosting providers like http://render.com/ who have a (somewhat modest) free tier for Docker instances, or reasonably priced ones like fly.io https://fly.io/docs/about/pricing/.

[ Parent | on HN ]
Re:Show HN: Pathway – Build Mission Critical ETL and RAG in ... by threecheese on Friday June 14, 2024 @07:18PM (#40686077)
That’s a very interesting model, I don’t think I’ve seen that before. Is the rust engine that sits underneath Python shipped as a compiled executable, with a license check/capability limitation? EDIT: there is rust source code in the Pathway repository.

Another edit: there is license checking code in the rust source; it seems fair to ask users of your copyrighted code to abide by your limits, even if they are self-enforced, if that’s the implicit agreement in the sharing of the thing. Objectively.

[ Parent | on HN ]
Re:Show HN: Pathway – Build Mission Critical ETL and RAG in ... by dxtrous on Sunday June 16, 2024 @06:11AM (#40695952)
It's absolutely great to see you've figured out the details of it! Indeed, the repository comes with a mix of Python and Rust which need to be built together. We trust our users not to re-build the package with altered parameters for production use (given the package build from sources is slightly non-trivial and takes an hour or two, one cannot really get this wrong by accident...). Then, for learning and non-production use, the BSL under which Pathway is distributed, allows you to do almost anything with the code.
[ Parent | on HN ]