News for hackers. Stuff that matters.
Re:Instant Cloud – SSD Bare Metal Servers by growt on Friday April 10, 2015 @05:52AM (#9353330)
I made a habit out of measuring the disk performance of cloud servers, so here it is:

  ubuntu@instantcloud:~$ dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync                                                                         
  1024+0 records in                                                                                                                                      
  1024+0 records out                                                                                                                                     
  1073741824 bytes (1.1 GB) copied, 11.0101 s, 97.5 MB/s  

  ubuntu@instantcloud:~$ sudo hdparm -tT /dev/nbd0                                                                                                       
                                                                                                                                                       
  /dev/nbd0:                                                                                                                                             
   Timing cached reads:   2180 MB in  2.00 seconds = 1090.44 MB/sec                                                                                      
   Timing buffered disk reads: 268 MB in  3.02 seconds =  88.85 MB/sec
Re:Instant Cloud – SSD Bare Metal Servers by venti on Friday April 10, 2015 @05:55AM (#9353340)
How does that compare to other cloud servers?
Re:Instant Cloud – SSD Bare Metal Servers by rshm on Friday April 10, 2015 @06:07AM (#9353383)
Tested on random instance, without any io intensive application running

VULTR 2 GB INSTANCE dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync

    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB) copied, 8.11451 s, 132 MB/s
    
    
    hdparm -tT /dev/vda
    
    /dev/vda:
    Timing cached reads:   25106 MB in  2.00 seconds = 12568.51 MB/sec
    Timing buffered disk reads: 496 MB in  3.02 seconds = 164.16 MB/sec

DIGITALOCEAN 2GB INSTANCE

    hdparm -tT /dev/vda
    
    Timing cached reads:   8926 MB in  2.00 seconds = 4468.52 MB/sec
    Timing buffered disk reads: 542 MB in  3.00 seconds = 180.43 MB/sec


    dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync 
    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB) copied, 32.0005 s, 33.6 MB/s
Re:Instant Cloud – SSD Bare Metal Servers by reidrac on Friday April 10, 2015 @07:12AM (#9353558)
I'm not sure how good is the test as it depends on the IO levels of the host where the VM instance is running.

This is a 1GB Debian Wheezy instance with regular HDD in Memset [1]:

    dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB) copied, 17.7142 s, 60.6 MB/s
I checked the host and its load is average. DO's 33.6 MB/s sounds poor.

[1] disclaimer: Memset hosting is my employer, I built some of this; but this is an honest test (personal VM running several services).

EDIT: formatting

EDIT 2: same test on a VM with SSD disk:

    dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB) copied, 3.48953 s, 308 MB/s
Again, the host has average load.
Re:Instant Cloud – SSD Bare Metal Servers by antouank on Friday April 10, 2015 @06:34AM (#9353452)
Did the same test on a VM I have in DigitalOcean... ( Ubuntu, 512 MB droplet )

~# dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 2.35872 s, 455 MB/s

Re:Instant Cloud – SSD Bare Metal Servers by hobarrera on Sunday April 12, 2015 @03:49PM (#9364527)
Add four spaces at the start of a line to mark it as code (both to avoid lines being merged and fixed-width font).

(yes: it's just markdown here).

Re:Instant Cloud – SSD Bare Metal Servers by growt on Friday April 10, 2015 @06:03AM (#9353367)
You can read my comment history for exact numbers. But for SSD storage it's not very good. Better than amazon EC2 though (t2 instance).
Re:Instant Cloud – SSD Bare Metal Servers by edouardb on Friday April 10, 2015 @06:10AM (#9353389)
You should have a look to the IOPS instead bandwidth

  root@instantcloud:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=16 --size=4G --readwrite=randwrite
  test: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
  fio-2.1.11
  Starting 1 process
  Jobs: 1 (f=1): [w(1)] [100.0% done] [0KB/18714KB/0KB /s] [0/4678/0 iops] [eta 00m:00s]

  root@instantcloud:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=16 --size=4G --readwrite=randread
  test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
  fio-2.1.11
  Starting 1 process
  Jobs: 1 (f=1): [r(1)] [100.0% done] [22307KB/0KB/0KB /s] [5576/0/0 iops] [eta 00m:00s]
Re:Instant Cloud – SSD Bare Metal Servers by buster on Friday April 10, 2015 @07:06AM (#9353542)
Didn't know about fio, thanks!
Re:Instant Cloud – SSD Bare Metal Servers by mrmondo on Friday April 10, 2015 @08:24AM (#9353802)
FYI - dd is not an effective way of measuring disk performance.

I'd recommend using fio to do this using libaio and direct disk reads/writes, and IOPing for basic latency tests:

* Random 4k read test for flash storage:

  fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
  --filename=test --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
  --readwrite=randread
* And writes:

  fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
  --filename=test --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
  --readwrite=randwrite
---

Here's an example here is a test storage unit I'm logged into at work right now (NOTE: THIS IS NOT ON INSTANTCLOUD!):

  root@s1-san5:~ # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
  --filename=/dev/md200 --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
  --readwrite=randread
test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128

...

  Run status group 0 (all jobs):
  READ: io=61440MB, aggrb=2339.7MB/s, minb=199652KB/s, maxb=200362KB/s, mint=26167msec, maxt=26260msec
  
  Disk stats (read/write):
  md100: ios=15636294/0, merge=0/0, ticks=0/0, in_queue=1581403800, util=100.00%, aggrios=7864320/0, aggrmerge=0/0, aggrticks=128880/0, aggrin_queue=131372, aggrutil=100.00%
  nvme0n1: ios=7576253/0, merge=0/0, ticks=123812/0, in_queue=125748, util=100.00%
  nvme1n1: ios=8152387/0, merge=0/0, ticks=133948/0, in_queue=136996, util=100.
While the test is running is you'll see the storage performance (Note the MB/s in the third [] and iops in the fourth []):

  Jobs: 12 (f=12): [r(12)] [12.3% done] [2537MB/0KB/0KB /s] [650K/0/0 iops] [eta 00m:50s]
And when it's completed:

* Throughput: aggrb=2339.7MB/s

* IOs (Read in this case): ios=15636294/0

And then with IOPing to test latency:

* Here's an example of really bad storage latency on my crappy old rotational RAID array at home:

  root@nas:/mnt/raid# ioping /dev/md0
  4 KiB from /dev/md0 (block device 7.28 TiB): request=1 time=27.2 ms
  4 KiB from /dev/md0 (block device 7.28 TiB): request=2 time=15.7 ms
* Here's an example of pretty good storage latency on my new storage at work:

  root@s1-san5:~  # ioping /dev/md200
  4 KiB from /dev/md200 (block device 1.09 TiB): request=1 time=136 us
  4 KiB from /dev/md200 (block device 1.09 TiB): request=2 time=124 us
  4 KiB from /dev/md200 (block device 1.09 TiB): request=3 time=112 us
* And one more on a VM with storage provisioned over iSCSI to a very slow rotational storage array that's quite busy:

  root@nagios:~ # ioping /dev/xvda
  4096 bytes from /dev/xvda (device 15.0 Gb): request=1 time=11.6 ms
  4096 bytes from /dev/xvda (device 15.0 Gb): request=2 time=0.2 ms
  4096 bytes from /dev/xvda (device 15.0 Gb): request=3 time=7.1 ms
  4096 bytes from /dev/xvda (device 15.0 Gb): request=4 time=1.2 ms
---

OK So let's try this on instantcloud.io / scaleway:

* IOP/s - Random 4k reads:

  bs: 12 (f=12): [r(12)] [0.5% done] [31050KB/0KB/0KB /s] [7762/0/0 iops] [eta 37m:32s]
* IOP/s - Random 4k writes:

  Jobs: 12 (f=12): [w(12)] [0.2% done] [0KB/7848KB/0KB /s] [0/1962/0 iops] [eta 02h:29m:10s]
* Latency:

   root@instantcloud:~# ioping /dev/nbd0
   4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=1 time=1.4 ms
   4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=2 time=1.4 ms
   4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=3 time=1.4 ms
   4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=4 time=2.8 ms

Conclusion:

Good performance for small arm devices but not even close to even a single entry-level consumer grade SATA SSD.

Re:Instant Cloud – SSD Bare Metal Servers by hobarrera on Sunday April 12, 2015 @03:48PM (#9364520)
That's rather poor, considering my laptop with FDE does about 50% faster:

    ▶ dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB) copied, 7.32253 s, 147 MB/s