News for hackers. Stuff that matters.
Re:A Gluster developer's thoughts on Torus by dice on Monday June 06, 2016 @06:18PM (#11850909)
>Single-threaded sequential 1KB writes for a total of 4GB, without even oflag=sync? Bonnie++? Sorry, but these are not "good benchmarks to run" at all. They're garbage. People who know storage would never suggest these.

As a non-storage person: what should I be using instead of dd and bonnie++?

Re:A Gluster developer's thoughts on Torus by notacoward on Monday June 06, 2016 @06:53PM (#11851148)
I usually recommend iozone and fio. Iozone isn't the most powerful tool, but it's simple to use and with the right options can at least provide some useful information. Fio is much more powerful, in terms of the workloads it can generate and the information it can provide, but it's a real PITA to work with.

As it turns out, I've given a few presentations on this stuff. Here's one of the more recent ones:

https://docs.google.com/presentation/d/1CQgn_fXivdXZ-fcqJ39v...

Re:A Gluster developer's thoughts on Torus by bcantrill on Monday June 06, 2016 @06:40PM (#11851059)
bonnie++ is, as Jeff says, well-known as a canonical example of a benchmark gone horribly wrong; see Brendan Gregg's excellent post on active benchmarking and bonnie++[1] for the gory details.

Beyond bonnie++, storage benchmarks are fraught with peril; years ago, I dismembered SPEC SFS as being similarly unsafe at any speed when benchmarking storage systems, albeit for much more subtle reasons than the glaring mechanical flaws in bonnie++.[2] And as for dd, I actually think it's okay as long as you explain clearly what it is (and isn't); Jeff's complaint is that they seem to be treating this single dd invocation as "write performance", when it fact the truth is subtler -- and things like blocksize and synchronicity matter a great deal.

More generally, anyone interested in storage benchmarks would be wise to read essentially everything that Brendan Gregg has written on the subject, starting with his five-part (!!) series on file system latency.[3][4][5][6][7]

[1] http://www.brendangregg.com/ActiveBenchmarking/bonnie++.html

[2] http://dtrace.org/blogs/bmc/2009/02/02/eulogy-for-a-benchmar...

[3] http://dtrace.org/blogs/brendan/2011/05/11/file-system-laten...

[4] http://dtrace.org/blogs/brendan/2011/05/13/file-system-laten...

[5] http://dtrace.org/blogs/brendan/2011/05/18/file-system-laten...

[6] http://dtrace.org/blogs/brendan/2011/05/24/file-system-laten...

[7] http://dtrace.org/blogs/brendan/2011/06/03/file-system-laten...

Re:A Gluster developer's thoughts on Torus by notacoward on Monday June 06, 2016 @06:55PM (#11851156)
+1 for anything Brendan writes or says. Seriously, to those of us who do this stuff, he's idol material.
Re:A Gluster developer's thoughts on Torus by SEJeff on Monday June 06, 2016 @11:34PM (#11852217)
Ditto for Mr Cantrill, one of the fathers of DTrace fwiw
Re:A Gluster developer's thoughts on Torus by sijoe on Tuesday June 07, 2016 @12:46PM (#11855702)
TL;DR version: Anything that is not your application won't use the resources in the same way, and will have very different properties (scaling, performance, contention, etc.)

Longer:

bonnie++ is not a load generator. Really. Start a typical run with a command line test case you find online and you'll see no actual IO. Lots of cache hits. We used to use it, 10 years ago, to play around with load generation, but found that it didn't generate enough IO, or in a way that actually matched what people do.

IOzone is marginal ... I wouldn't use it for a serious test, and when people suggest dd or IOzone, I ask them how well the code actually matches their use case. Chances are that it is also largely irrelevant to this. Worse still is that the IOzone throughput measurements are basically bogus, using a naive sum of bandwidths, rather than showing the interesting data (the actual histogram or distribution of performance, including the start/end times, and the rates per thread/process as a function of time).

fio is good, in that you can implement many types of tests that have a reasonable chance at being meaningful. I caught Sandforce controllers compressing non-random data on benchmarks they were using for the SSDs that used them with fio. Actual SF performance was lower than spinning rust once you fed it real random data.

We wrote something called io-bm (https://gitlab.scalableinformatics.com/joe/io-bm) for pounding on parallel file systems (specifically to stress them and see how they scaled and dealt with contention for network, metadata, etc.). I am not sure if the repo has our histogramming and time series bits in it so we can see individual thread performance as well as overall performance, might be in a private repo.

Basically it boils down to the TL;DR above. If its not your code, then you are likely testing with an application that is somewhere between partially to wholly irrelevant to your use case.

Re:A Gluster developer's thoughts on Torus by Terribledactyl on Monday June 06, 2016 @06:35PM (#11851026)
There are two main issues with generalized benchmarking.

1) You (probably) aren't testing the things most done by your application/platform/etc. And worse yet, the hardware/software maybe simply going for big numbers in the common benchmarks.

2) The test may or may not be garbage depending on your use cases.

Re:A Gluster developer's thoughts on Torus by subway on Monday June 06, 2016 @06:30PM (#11850993)
fio is a favorite of mine.

https://github.com/axboe/fio

Re:A Gluster developer's thoughts on Torus by bogomipz on Tuesday June 07, 2016 @07:46AM (#11853771)
Re:A Gluster developer's thoughts on Torus by SEJeff on Monday June 06, 2016 @06:33PM (#11851012)
fio, maintained by Jens Axboe, the guy who maintains the Linux kernel block layer.
Re:A Gluster developer's thoughts on Torus by wildlogic on Tuesday June 07, 2016 @03:40AM (#11852988)
Intel developed tool - http://www.iometer.org/
Re:A Gluster developer's thoughts on Torus by DeepYogurt on Monday June 06, 2016 @07:00PM (#11851184)
CompileBench is a decent one. It's made by the author of btrfs. https://oss.oracle.com/~mason/compilebench/