VULTR 2 GB INSTANCE dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 8.11451 s, 132 MB/s
hdparm -tT /dev/vda
/dev/vda:
Timing cached reads: 25106 MB in 2.00 seconds = 12568.51 MB/sec
Timing buffered disk reads: 496 MB in 3.02 seconds = 164.16 MB/sec
DIGITALOCEAN 2GB INSTANCE hdparm -tT /dev/vda
Timing cached reads: 8926 MB in 2.00 seconds = 4468.52 MB/sec
Timing buffered disk reads: 542 MB in 3.00 seconds = 180.43 MB/sec
dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 32.0005 s, 33.6 MB/sThis is a 1GB Debian Wheezy instance with regular HDD in Memset [1]:
dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 17.7142 s, 60.6 MB/s
I checked the host and its load is average. DO's 33.6 MB/s sounds poor.[1] disclaimer: Memset hosting is my employer, I built some of this; but this is an honest test (personal VM running several services).
EDIT: formatting
EDIT 2: same test on a VM with SSD disk:
dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.48953 s, 308 MB/s
Again, the host has average load.~# dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 2.35872 s, 455 MB/s
(yes: it's just markdown here).
root@instantcloud:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=16 --size=4G --readwrite=randwrite
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.11
Starting 1 process
Jobs: 1 (f=1): [w(1)] [100.0% done] [0KB/18714KB/0KB /s] [0/4678/0 iops] [eta 00m:00s]
root@instantcloud:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=16 --size=4G --readwrite=randread
test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.11
Starting 1 process
Jobs: 1 (f=1): [r(1)] [100.0% done] [22307KB/0KB/0KB /s] [5576/0/0 iops] [eta 00m:00s]I'd recommend using fio to do this using libaio and direct disk reads/writes, and IOPing for basic latency tests:
* Random 4k read test for flash storage:
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
--filename=test --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
--readwrite=randread
* And writes: fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
--filename=test --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
--readwrite=randwrite
---Here's an example here is a test storage unit I'm logged into at work right now (NOTE: THIS IS NOT ON INSTANTCLOUD!):
root@s1-san5:~ # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
--filename=/dev/md200 --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
--readwrite=randread
test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128...
Run status group 0 (all jobs):
READ: io=61440MB, aggrb=2339.7MB/s, minb=199652KB/s, maxb=200362KB/s, mint=26167msec, maxt=26260msec
Disk stats (read/write):
md100: ios=15636294/0, merge=0/0, ticks=0/0, in_queue=1581403800, util=100.00%, aggrios=7864320/0, aggrmerge=0/0, aggrticks=128880/0, aggrin_queue=131372, aggrutil=100.00%
nvme0n1: ios=7576253/0, merge=0/0, ticks=123812/0, in_queue=125748, util=100.00%
nvme1n1: ios=8152387/0, merge=0/0, ticks=133948/0, in_queue=136996, util=100.
While the test is running is you'll see the storage performance (Note the MB/s in the third [] and iops in the fourth []): Jobs: 12 (f=12): [r(12)] [12.3% done] [2537MB/0KB/0KB /s] [650K/0/0 iops] [eta 00m:50s]
And when it's completed:* Throughput: aggrb=2339.7MB/s
* IOs (Read in this case): ios=15636294/0
And then with IOPing to test latency:
* Here's an example of really bad storage latency on my crappy old rotational RAID array at home:
root@nas:/mnt/raid# ioping /dev/md0
4 KiB from /dev/md0 (block device 7.28 TiB): request=1 time=27.2 ms
4 KiB from /dev/md0 (block device 7.28 TiB): request=2 time=15.7 ms
* Here's an example of pretty good storage latency on my new storage at work: root@s1-san5:~ # ioping /dev/md200
4 KiB from /dev/md200 (block device 1.09 TiB): request=1 time=136 us
4 KiB from /dev/md200 (block device 1.09 TiB): request=2 time=124 us
4 KiB from /dev/md200 (block device 1.09 TiB): request=3 time=112 us
* And one more on a VM with storage provisioned over iSCSI to a very slow rotational storage array that's quite busy: root@nagios:~ # ioping /dev/xvda
4096 bytes from /dev/xvda (device 15.0 Gb): request=1 time=11.6 ms
4096 bytes from /dev/xvda (device 15.0 Gb): request=2 time=0.2 ms
4096 bytes from /dev/xvda (device 15.0 Gb): request=3 time=7.1 ms
4096 bytes from /dev/xvda (device 15.0 Gb): request=4 time=1.2 ms
---OK So let's try this on instantcloud.io / scaleway:
* IOP/s - Random 4k reads:
bs: 12 (f=12): [r(12)] [0.5% done] [31050KB/0KB/0KB /s] [7762/0/0 iops] [eta 37m:32s]
* IOP/s - Random 4k writes: Jobs: 12 (f=12): [w(12)] [0.2% done] [0KB/7848KB/0KB /s] [0/1962/0 iops] [eta 02h:29m:10s]
* Latency: root@instantcloud:~# ioping /dev/nbd0
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=1 time=1.4 ms
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=2 time=1.4 ms
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=3 time=1.4 ms
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=4 time=2.8 ms
Conclusion:Good performance for small arm devices but not even close to even a single entry-level consumer grade SATA SSD.
▶ dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 7.32253 s, 147 MB/s