News for hackers. Stuff that matters.
Re:Datomic Pro Starter Edition by Jemaclus on Friday November 08, 2013 @02:04PM (#6698380)
Companies really need to do a better job of making ELI5 descriptions. This means nothing to me:

> Datomic is a database of flexible, time-based facts, supporting queries and joins, with elastic scalability, and ACID transactions. > Datomic can leverage highly-available, distributed storage services, and puts declarative power into the hands of application developers.

What does that even mean? I'm pretty good at databases, and I know what transactions and queries and joins and stuff, but this explanation does a pretty poor job of explaining what Datatomic is and why I should use it.

What's a "time-based fact"? What does putting "declarative power in the hands of application developers" mean? Don't all databases support queries? Don't most relational databases support joins and transactions?

It just seems like a buzz-word filled fluff phrase that says "Datatomic is a database," but I don't know why I would use it over, say, MySQL or Mongo or whatever.

I had to go to Cognitect's site (http://cognitect.com/datomic) to get a better explanation:

>Datomic is built on immutable data; facts are never forgotten or overwritten. This means complete auditability is built in from the start - including the transactions that modify your database. And because Datomic is built on immutable data, you can explore possible future scenarios by issuing transactions against a database and decide to commit them only after verifying the results.

Ok, that makes a little more sense, but it's still not clear what differentiates this from other database systems.

I'm not affiliated with them at all, but I think SiftScience.com does a great job of explaining their product simply and efficiently: https://siftscience.com/

> Fight Fraud with Machine Learning > Sift Science monitors your site's traffic in real time and alerts you instantly to fraudulent activity.

Simply detects fraud. Done.

Or EasyPost.com:

> Shipping for developers > EasyPost allows you to integrate shipping APIs into any application in minutes.

Don't just spit buzz words and technical terms at me. Tell me what it does.

> Datomic is a database that, among other things, specializes in tracking data over time, allowing you to test a transaction before saving the data. True accountability from the beginning!

Probably not even accurate (again, I don't understand Datomic's premise), but that sounds more explanatory than their current buzz word-filled blurb.

To any startups watching: have something on your front page that is an ELI5 description of your product.

Re:Datomic Pro Starter Edition by brandonbloom on Friday November 08, 2013 @02:26PM (#6698509)
You're right, their description should be clearer, or have an elaboration or something. But for onlookers:

> flexible

Minimal, non-tabular schema

> time-based

All historical values are preserved.

> facts

Triple store: Entity / Attribute / Value

with time:

Quad store: Entity / Attribute / Value / TIME

> supporting queries and joins

Datalog query language

> with elastic scalability

Reads are uncoordinated, so they scale horizontally

> ACID transactions.

You can trust the database won't lose stuff you write to it.

> leverage highly-available, distributed storage services

Storage engines are pluggable. You can choose backends such as filesystems, Riak, Dynamo, traditional SQL databases, etc.

> puts declarative power into the hands of application developers.

Reads occur in-process, like an embedded database. Queries, traversal, and other reads occurs locally, without network round trips.

Re:Datomic Pro Starter Edition by tony_landis on Friday November 08, 2013 @04:12PM (#6699198)
Generally speaking, five year olds should not be choosing databases for anything critical. But I agree that Datomic and company can do better with their marketing and general product description.

Here is one of the better explanations I have seen:

http://www.flyingmachinestudios.com/programming/datomic-for-...

Re:Datomic Pro Starter Edition by cordite on Friday November 08, 2013 @02:50PM (#6698667)
There's definitely a lot more to ACID than the part you mention, which fulfills the D or Durability aspect.

Though to any competent database engineer, they already know what ACID is.

The videos on datomic's website help clarify the understanding, but that isn't as accessible as well written text.

Re:Datomic Pro Starter Edition by brandonbloom on Friday November 08, 2013 @03:16PM (#6698829)
> (map eli5 (acronym-expand "ACID"))

Atomic: Transactions are all-or-nothing. You never see half of a transaction. You either get the world before the transaction, or after it.

Consistent: The before and after worlds are always valid, never corrupt.

Isolated: From the outside, it appears that transactions occur one-after-another, never overlapping in time. In Datomic, this is actually the case: Transactions are fully serialized.

Durability: If a transaction succeeds, you can be confident that the consistent "after" world is safely on disk.

Re:Datomic Pro Starter Edition by Jemaclus on Friday November 08, 2013 @03:11PM (#6698794)
Thanks for breaking down the buzzwords into something understandable!
Re:Datomic Pro Starter Edition by puredanger on Friday November 08, 2013 @02:22PM (#6698494)
"Time-based fact" = entity-attribute-value at time. Example: Alex likes nachos at 5 pm.

"Declarative power" = among other things, datalog queries. SQL is also relatively declarative. Datomic has the benefit of declaring queries as data, not as munged strings.

"Facts are never forgotten or overwritten" = as the world changes, add a new record at a new timestamp. Traditional relational dbs modify the old record destroying previous values. The database notion of time (series of transactions) is made explicit and available as part of the query (query the db at a particular point in time or even join databases from different points in time).

Re:Datomic Pro Starter Edition by cordite on Friday November 08, 2013 @02:07PM (#6698399)
I had to look up what ELI5 meant :)

For any others that are wondering, "Explain Like I'm 5"

Re:Datomic Pro Starter Edition by espeed on Friday November 08, 2013 @05:26PM (#6699546)
Here's the new site: http://cognitect.com/datomic

And Datomic has been discussed in detail here...

* Rich Hickey on Datomic: Datalog, Databases, Persistent Data Structures (http://news.ycombinator.com/item?id=3797272)

* Rich Hickey on Datomic, CAP and ACID (http://news.ycombinator.com/item?id=5093037)

* The Architecture of Datomic (https://news.ycombinator.com/item?id=4732524)

* Datomic Free Edition (http://news.ycombinator.com/item?id=4286121)

* Thinking in Datomic (https://news.ycombinator.com/item?id=4215765)

* Datomic: Can Simple be also Fast? (https://news.ycombinator.com/item?id=6426362)

Docs: http://docs.datomic.com

UPDATE: Jernau gives a great practical overview in his screencast (https://www.youtube.com/watch?v=ao7xEwCjrWQ) on building a basic app with Datomic, Clojure, and Light Table.

Also see "Learn Datalog Today" (http://www.learndatalogtoday.org) for an interactive Datalog tutorial.

Re:Datomic Pro Starter Edition by Jemaclus on Monday November 11, 2013 @02:33PM (#6713443)
Why would I waste my time reading all of those articles or watching a screencast, if I don't have any idea what Datomic is? In order for me to be interested enough to click any of those links, I would have to think Datomic is a) interesting, b) relevant to me, or c) worth my time. The current website doesn't accomplish any of those objectives. That's what i'm getting at.

I'm sure it's a wonderful app, but its first-impression presentation doesn't tell me what it is I'm looking at. And as such, I'm just going to click away and find something else.

Re:Datomic Pro Starter Edition by coolsunglasses on Friday November 08, 2013 @02:21PM (#6698481)
Reproducibility of results from the canonical data store thanks to having the full history of everything is pretty great.

"Code can never corrupt data again"

Re:Datomic Pro Starter Edition by rdtsc on Friday November 08, 2013 @10:12PM (#6700507)
In theory sounds good but this is just not feasible.

Couple of scenarios:

1) A runaway data generator. Somehow you end up confusing milliseconds with seconds and now you start logging 1000x more data than you intended, you leave for the weekend and find your storage is full of junk

2) Sensitive data. Felix has just inserted all the passwords and credit card numbers in the database.

Re:Datomic Pro Starter Edition by puredanger on Friday November 08, 2013 @10:19PM (#6700542)
Datomic offers excision to cover cases like these (or other common cases where you are required by law to forget things). http://blog.datomic.com/2013/05/excision.html
Re:Datomic Pro Starter Edition by coolsunglasses on Saturday November 09, 2013 @12:00AM (#6700736)
Do you assume by default that people can't anticipate obvious things like this in general or only when it's talented programmers working on a database?
Re:Datomic Pro Starter Edition by rdtsc on Sunday November 10, 2013 @02:17AM (#6705625)
> or only when it's talented programmers working on a database?

I'll pretend I don't know who Hickey is and just say that I expect a lot of stupid decisions can be made when coming up with "a completely new way of looking or storing data". Some databases (cough...cough...starts with M) shipped their database product with unacknowledged writes as the default, oops. Everyone can make a database.

Now let's dig deeper. The reason I asked what I asked it because given the architecture of Datomic and the claims made, that corner case (excision) would not be trivial or and would have to be implemented as a hack on the side. That is why I asked. In some databases it is easier to do, in that architecture it is hard.

> or only when it's talented programmers working on a database?

Who are these talented programmers? Are you a talented programmer?

Re:Datomic Pro Starter Edition by hammerandtongs on Friday November 08, 2013 @02:19PM (#6698467)
Re:Datomic Pro Starter Edition by lazylland on Friday November 08, 2013 @02:54PM (#6698689)
what's really worrying is that in all this time I've never seen a convincing benchmark, to motivate diving 'head first' into Datomic. For all the hate that the MongoDB guys get, they alteast piqued interest with some arresting charts and figures.

Right now, the impression I get is, "ooh, because Rich Hickey..."

Could anyone share anecdotes to the counter ?

Re:Datomic Pro Starter Edition by VaedaStrike on Friday November 08, 2013 @03:12PM (#6698799)
I'm self-taught at Relational Database Design. I spent two years worth of about all of my free time learning, and creating, large normalized SQL schemas. One of the things that most struck me was the capacity to get virtually all the advantages of a relational database without the baggage and rigidity. Add to that the fact that you have a distributed database with almost none of the baggage usually associated with such was another big thing for me.

Granted I'm coming from a do-it-myself green-field project, but the things that Datomic's structure and design opens up for my personal project mean that me, a one man show, can far more easily go toe to toe with the big guys with massive teams to design and deploy their apps and databases.

Potential is what I see, and it's far more than I've seen with just about any other technology that I can think of. I really think Datomic is a game changer, at least for the few awake enough to see it.

Re:Datomic Pro Starter Edition by AlisdairO on Friday November 08, 2013 @03:23PM (#6698877)
Datomic is a very cool project, but it's worth being aware that triple stores generally exhibit significantly worse performance than relational DBs for most query workloads. For non-huge projects they're perfectly capable though, and do offer some flexibility benefits.
Re:Datomic Pro Starter Edition by puredanger on Friday November 08, 2013 @03:41PM (#6698998)
Please note that Datomic's architecture is substantially different than most traditional triple stores so I don't think it is useful to extrapolate performance for one from the other.
Re:Datomic Pro Starter Edition by AlisdairO on Friday November 08, 2013 @05:13PM (#6699490)
That's true, and if your workload is such that you're able to predictably query over a relatively small fraction of the database, then I can imagine it working well - it effectively buys you auto-partitioning. If your queries run over a larger subsection of the data, you're still going to run into the fundamental problem of most triple stores: they're join and random i/o heavy. I've not heard that Datomic has anything special going on with regards to this issue, although I may be out of date.

This isn't intended to bag on triple stores - I work on them for a living - or Datomic specifically. Triple stores are very useful and in many ways liberating to work with, but they do present challenges when querying over lots of data.

Re:Datomic Pro Starter Edition by VaedaStrike on Friday November 08, 2013 @04:10PM (#6699183)
I'd just eco PureDanger's comment regarding the architecture. I'm no expert on the matter but the concept of having the components of the database taken apart, as they are with Datomic, would give you potential arrangements that you couldn't get with traditional architectures.
Re:Datomic Pro Starter Edition by jeletonskelly on Friday November 08, 2013 @03:40PM (#6698991)
This is why datomic's query magic is in it's indexes.
Re:Datomic Pro Starter Edition by lazerwalker on Friday November 08, 2013 @03:16PM (#6698828)
I haven't actually used Datomic yet, but what piques my interest isn't necessarily "ooh, Rich Hickey" as much as "ooh, the Clojure/functional approach to data immutability applied to a database". The appeal, to me at least, is in the potential to make my life easier as a product engineer, rather than as a DBA or devops guy.
Re:Datomic Pro Starter Edition by jeletonskelly on Friday November 08, 2013 @03:37PM (#6698969)
Very much one of the biggest draws in my opinion, as I've been using Datomic in production for several months. We're a clojure shop and one of the things I've been working on recently is developing a library for parsing/storing/querying all of clojure's data structures into datoms. I hope I get to open source it :)
Re:Datomic Pro Starter Edition by espeed on Friday November 08, 2013 @06:40PM (#6699854)
See "Datomic: Can Simple be also Fast?"(https://news.ycombinator.com/item?id=6426362)
Re:Datomic Pro Starter Edition by etherealG on Friday November 08, 2013 @05:05PM (#6699450)
valuable software isn't always easy and quick to pick up. simple things can be difficult to familiarise yourself with.

learning the value of functional style took ages for me to grok, but it's been one of the biggest values ever.

i find it a little ironic considering one of the first and best talks i've seen from hickey is 'simple made easy'. you'd likely learn a lot from a watch of that, or perhaps more detail about datomic in any of the talks on datomic.

Re:Datomic Pro Starter Edition by stonemetal on Friday November 08, 2013 @02:47PM (#6698656)
It is a NoSQL database with a novel architecture. It uses schemas, supports transactions, historical queries, and it doesn't use a client server model.
Re:Datomic Pro Starter Edition by Jemaclus on Friday November 08, 2013 @03:10PM (#6698789)
Now this I understand. Thanks.