News for hackers. Stuff that matters.
Re:Eva – A distributed entity-attribute-value database in Cl... by CurrentB on Friday June 28, 2019 @10:42AM (#20304759)
I'm excited that some of the ideas from Datomic are starting to make it into some open source projects. With crux last month and now Eva, there are now multiple options for modern clojure databases.

While there are many excellent ideas embedded in Datomic and these projects, for me just being able to persist the same data structures you're using at a repl and query for them with data is a huge win vs having to start translating types and concepts and query strings to and from SQL is a huge win.

Re:Eva – A distributed entity-attribute-value database in Cl... by derefr on Friday June 28, 2019 @11:13AM (#20305115)
> I'm excited that some of the ideas from Datomic are starting to make it into some open source projects

Now if only they’d make it into open source projects in languages other than Clojure. I want my Datomic-for-Elixir, darn it! (And, if I wasn’t too busy, I’d be the first person to volunteer to build it!)

Re:Eva – A distributed entity-attribute-value database in Cl... by skrebbel on Friday June 28, 2019 @12:11PM (#20305829)
It's not precisely the same thing, but I wonder why the Elixir community doesn't seem to use mnesia too much. It ships right there with the OTP and you can store any erlang term.
Re:Eva – A distributed entity-attribute-value database in Cl... by derefr on Friday June 28, 2019 @12:51PM (#20306288)
Mnesia works great for what it’s designed for. What it was designed for, though, is “replicating connection state between a hot-master telecom switch and its warm-standby switch, so that the warm standby can be instantly promoted to master.” Which... isn’t a common use-case, really. I’d describe the result as “ETS synchronization”, rather than a true DBMS.

Mnesia has had features like disc copies bolted on after-the-fact, but you can tell from the way they’re implemented (e.g. manual node-crash recovery) that they’re not used by Ericsson in the form you find them in OTP, but rather that these are just framework hooks where Ericsson has built (and expects you to also build) specialized-to-your-use-case DBMS logic on top of.

And, likewise, Mnesia assumes OTP’s “distribution set” model (i.e. a static set of known operational relationships between nodes) rather than a clustering model; Mnesia has no clustering support per se—you need to take down the whole Mnesia application across the distribution set and start it back up if you want to change its node membership.

These things are fixable, but the result wouldn’t be really be “Mnesia” itself any more, but rather a freestanding DBMS system that uses Mnesia as its storage- and transaction-linearization engine, maybe Lasp for clustering, CRDT-annotated field-types in table schemas for resolving state after crashes, etc. Still, I’m surprised nobody has bothered to build such a thing and open-source it.

Re:Eva – A distributed entity-attribute-value database in Cl... by lostcolony on Friday June 28, 2019 @02:14PM (#20307242)
Just as clarification, if you want a node to join a different cluster you have to take down the Erlang VM and restart it. If you want it to just rejoin the cluster you don't (i.e., you have to change the cookie to match that of the cluster you want to join). But given a set cookie, and service discovery, you can join or leave a cluster without issue.

That said, I generally agree with you. Mnesia is perfectly serviceable, but it is, as you say, really only a good fit for manual intervention in the event of service interruption. That manual intervention can be after the fact, but it can't automatically resolve a partition in the event of a netsplit. It's not too bad to extend to write code to automatically resolve those partitions (I've POCed it; we ended up with a manual button to execute the code to do it just because of our own concerns, but it worked quite well), but you have to start making some decisions, and write some code, on how to do that.

Which is why people tend to use other things; there are other solutions that have "what do I do in the event of a netsplit" already built in. And while they may be faulty under certain circumstances (as Jepsen tests have shown is usually the case), most people will take mostly correct self healing over no/DIY self healing.

A few other things; Erlang's default distribution (which Mnesia leverages) was not built for remote distribution (i.e., nodes located in different data centers), so no clue what happens there. It also was built with a fully connected topology, which limits how many nodes it's reasonable to connect to the cluster.

Re:Eva – A distributed entity-attribute-value database in Cl... by ilikehurdles on Friday June 28, 2019 @11:27AM (#20305284)
Which databases are you currently using that are written in Elixir?
Re:Eva – A distributed entity-attribute-value database in Cl... by derefr on Friday June 28, 2019 @11:32AM (#20305349)
Of course, I meant Erlang; these things (Riak, CouchDB, RabbitMQ) always end up getting written in Erlang.

(Though I don’t think there’s been a new major infra component project started since Elixir became a viable contender, so maybe that could change.)

Re:Eva – A distributed entity-attribute-value database in Cl... by ilikehurdles on Friday June 28, 2019 @12:48PM (#20306257)
I didn't realize those were written in Erlang! Similarly, on the clojure side, Datomic and Crux I think are both written in Java, despite generally being consumed by clojurey interfaces.
Re:Eva – A distributed entity-attribute-value database in Cl... by estsauver on Friday June 28, 2019 @04:58PM (#20308736)
Crux is definitely written in Clojure and Hickey has said Datomic is written in Clojure in the past.
Re:Eva – A distributed entity-attribute-value database in Cl... by maximente on Friday June 28, 2019 @11:11AM (#20305090)
can't agree enough! was really interested in the now defunct mentat project, but was written in Rust - it's a game changer to have a JVM-powered datomic-ish project with time as a first class citizen.
Re:Eva – A distributed entity-attribute-value database in Cl... by refset on Friday June 28, 2019 @12:08PM (#20305787)
> time as a first class citizen

It's worth noting that time now has two levels of meaning in this context - transaction time and valid time.

Disclosure: working on Crux

Re:Eva – A distributed entity-attribute-value database in Cl... by maximente on Saturday June 29, 2019 @12:30PM (#20313424)
hey, thanks for the reply. could you highlight the differences between Eva and Crux as you see them, today?

i do like passing in a timestamp and getting the state of the world as Crux seems to implement, whereas Eva seems slightly less oriented on time based querying, more so for a sort of log setup (where they have EAV+T+added?)

Re:Eva – A distributed entity-attribute-value database in Cl... by refset on Saturday June 29, 2019 @01:04PM (#20313602)
Well, Eva overlaps very closely with Datomic so I would recommend taking a look at the FAQ entry I wrote on the Crux/Datomic comparison (with 8 sections covering the major differences!): https://juxt.pro/crux/docs/faq.html#_comparisons

`EAV+T+added?` is the canonical data representation used by Eva, whereas Crux relies on two discrete "document" and transaction logs as the canonical representation. There are many reasons for this (data eviction, "unbundled" scalability etc.) but bitemporality (i.e. the ability to transact into the past/future) is definitely the primary reason why Crux doesn't follow the `EAV+T+Added?` pattern. You can get a feel for the indexes Crux uses here: https://github.com/juxt/crux/blob/master/src/crux/codec.clj#...

Re:Eva – A distributed entity-attribute-value database in Cl... by eternalban on Friday June 28, 2019 @11:15AM (#20305135)
I suspect by "ideas from Datomic" you are referring to ideas predating Datomic (and Clojure) that are used and possibly refined in Datomic.
Re:Eva – A distributed entity-attribute-value database in Cl... by emmanueloga_ on Friday June 28, 2019 @03:32PM (#20308026)
To be fair, a quick google search for "open source datalog database" does not return much that is usable out-of-the-box, other than datomic: mostly academic projects or adapters for other databases with some different querying API, like SPARQL.

Datomic seems to be one of the best ones in terms of execution of the ideas behind it and usage in real world projects, to say the least.

Re:Eva – A distributed entity-attribute-value database in Cl... by abc_lisper on Friday June 28, 2019 @12:17PM (#20305897)
I think he meant to say they were not mainstream
Re:Eva – A distributed entity-attribute-value database in Cl... by bitwize on Friday June 28, 2019 @11:34AM (#20305382)
In the Clojure community, no new technology officially exists until Rich Hickey invents it.
Re:Eva – A distributed entity-attribute-value database in Cl... by fnordsensei on Friday June 28, 2019 @06:04PM (#20309141)
My impression is that the Clojure community (including Rich Hickey) is generally transparent and proud about digging up and reusing ideas from the 80s in a modern context.
Re:Eva – A distributed entity-attribute-value database in Cl... by ngcc_hk on Friday June 28, 2019 @04:46PM (#20308657)
Can you elaborate a bit more about their relationship?