A Bunch of Money on AWS and Some Benchmark Results

Re:A Bunch of Money on AWS and Some Benchmark Results by dkhenry on Tuesday May 14, 2019 @09:23PM (#19915538)

That "Think Time" that you are referring to is supposed to emulate users running transactions on the database. So its not the database waiting its the driver waiting. While I do understand the reason for putting that in, you know very well that violating that limit doesn't artificially give CockroachDB or MemSQL an advantage when you are talking about 100,000 warehouses and a random distribution off transactions.

If CockroachDB is concerned about THINK TIME enough to ask for the numbers to be removed, this would be a great opportunity for them to remove that limit and see exactly how much they could push the benchmark.

[ Parent | on HN ]

Re:A Bunch of Money on AWS and Some Benchmark Results by knz42 on Tuesday May 14, 2019 @09:34PM (#19915598)

You should read up on why this think time exists. It has nothing to do with "emulating slow clients" and everything to do with not claiming "I have a fast database" by running gazillion txn/sec on just 1MB of data in RAM.

TPC-C requires that you increase the amount of "live data" if you want to display/advertise more performance. That's the benchmark's rule.

If you want to benchmark something else, that's fine, but then

1) don't call it "TPC-C" 2) don't compare with databases that play by the rules.

[ Parent | on HN ]

Re:A Bunch of Money on AWS and Some Benchmark Results by dkhenry on Tuesday May 14, 2019 @10:09PM (#19915802)

I know why it exists, and my point is they are not trying to show a gazillion txn/sec on 1MB of data. You are looking at a dataset that is several TB's. They have far surpassed the point where a vendor is trying to cheat by putting all the data in L1 cache and claiming to be fast.

[ Parent | on HN ]