Re:Uncurled – running and maintaining Open Source projects f...
by 42jd on Wednesday May 18, 2022 @02:05AM
(#31419699)
I’m not sure how one could enable others to contribute easily but just making the code open source as is would be interesting. Those with a passing interest in search engines can browse the code of a capable engine and pick up some knowledge. While those who really want to help out will find a way to run it and contribute (in the process helping document and maybe making it easier).
My philosophy to open source is just do my personal projects in the open even if it’s not “accessible” and if someone comes along and likes it they can contribute or fork it and modify to their liking. Or they might just browse the code and pick up something in passing.
Re:Uncurled – running and maintaining Open Source projects f...
by closedloop129 on Tuesday May 17, 2022 @04:24PM
(#31415862)
Have you asked that bunch of people how they would like to cooperate?
My ignorant suggestion towards sharing those gigabytes of data: have you considered bittorrents?
Re:Uncurled – running and maintaining Open Source projects f...
by nijave on Wednesday May 18, 2022 @07:17PM
(#31428799)
Are those hard requirements and, if so, why? It might be worthwhile to add a "local" or "dev" mode which loads a very minimal amount of data (maybe only a handful of items are indexed and only a handful of search terms that work with them).
The problem is the logistics of an open search project that is a web service with serious hardware requirements. I think the minimum hardware requirements is about 14 Gb of RAM, that's without any real data loaded into the system. Testing is awkward and cumbersome, and the data logistics are a real headache even on the same network as the production instance.
To even run the search engine, you need a few hundred megabytes of language models, as well as a probably few gigabytes of website data to conduct meaningful testing. The production instance has a disk footprint of about half a terabyte.