The benchmarks do not use caliper or jmh. This is _very_ important - microbenchmarks on the JVM rarely do what you think they do without extra work.