From the license https://www.datomic.com/datomic-pro-edition-eula.html

> The Licensee hereby agrees ... it will not: ... (j) publicly display or communicate the results of internal performance testing or other benchmarking or performance evaluation of the Software;

This kind of clause is usually known as a "DeWitt clause"[1], after a University of Wisconsin professor who benchmarked a couple databases, including Oracle (which performed poorly). Oracle/Larry Ellison didn't react well to that and decided to forbid benchmarks.

[1]: https://danluu.com/anon-benchmark/

Database performance depends on so many variables - thread pools, queue sizes, RAM allocations to different purposes, disk layout - the list goes on. There are whole consulting fields that are essentially doing a random walk through database configuration files looking for better performance - ascending the performance gradient if you will.

So in this world, its sensible to say "benchmarks are uninformative and misleading - your mileage will almost certainly vary"

Benchmarks are easily abused, misused, and misinterpreted. E.g., benchmarks looking at some very specific aspect of query performance being extrapolated to more complex/real-world queries.

Also trade-offs are rarely mentioned in benchmark numbers– e.g., great write throughput, at the expense of: ?.

It's fun to be cynical about stuff like this, but it's rarely as simple as "Ellison didn't react well to that and decided to forbid benchmarks".

> Oracle/Larry Ellison didn't react well to that and decided to forbid benchmarks.

So you kind of have to wonder why Cognitect is going Oracle on us..

The most obvious explanation is that Datomic just doesn't perform well and they don't want people to know.

Anyone who has done serious performance testing on a DB knows that there's a massive gap between initial findings and a well tuned system designed with the help of the database maintainers. I've seen some nasty performance out of Riak, Cassandra, SQL, ElasticSearch etc. But with each of those, once I talked to the DB owners and fully understood the limitations of the system it was possible to make massive gains in performance.

Databases are complex programs, and if I ever wrote one, it would be infuriating for someone to pick it up, assume it was "just like MySQL" and then write a blog post crapping on it because it failed to meet their expectations.

Yes, benchmarks can give a misleading impression of a database's performance.

So what? Somehow PostgreSQL is doing fine despite that.

Which is worse publicity for Cognitect: people publishing bad benchmarks or Cognitect forbidding benchmarks Oracle style?

Well, the absurd one was:

>the license at least used to forbid even saying "Datomic" aloud in public.