Noms – A versioned, forkable, syncable database

Re:Noms – A versioned, forkable, syncable database by rspeer on Saturday October 15, 2016 @10:14PM (#12716505)

Can you point me at some? Because I've tried a few immutable data stores and been disappointed every time. Given about 10 GB of JSON structures, I keep finding things that can't outperform the boring combo of:

* Convert the versioned data to tab-separated values

* COPY it into Postgres every time

* Hope Postgres can act immutable enough even though it wasn't designed to be

The closest I've come to improving this situation was Kyoto Cabinet (unusable license) and rolling my own damn hashtable (it worked okay but adding new kinds of indexes was just unmaintainable, there's a reason databases should be made by experts).

[ Parent | on HN ]

Re:Noms – A versioned, forkable, syncable database by sjezewski on Sunday October 16, 2016 @12:39AM (#12716962)

(Disclaimer - I work at pachyderm)

http://pachyderm.io

Pachyderm is git for data. We work hard to make sure we can store data of different types (binary, text, json) efficiently. We also work hard to give you good mechanisms to read the data in a distributed way. I'd be curious how this suits your purposes.

[ Parent | on HN ]

Re:Noms – A versioned, forkable, syncable database by rspeer on Wednesday October 19, 2016 @07:54PM (#12748769)

Just started looking at Pachyderm.

While I can see how a git-based filesystem can help with some use cases, does it do any kind of indexing at all? I see that the FAQ recommends exporting the data from Pachyderm into PostgreSQL, which leaves me where I am now.

[ Parent | on HN ]

Re:Noms – A versioned, forkable, syncable database by ykler on Saturday October 15, 2016 @10:30PM (#12716558)

I was curious about Camlistore and Tahoe-LAFS at one point. I didn't investigate them very thoroughly, however. How do they compare?

[ Parent | on HN ]