News for hackers. Stuff that matters.
Re:Mkfile(8) is severely syscall limited on OS X by im3w1l on Sunday February 12, 2017 @07:04AM (#13627983)
Oh wow, I had no idea that write buffers needed to be on the order of a megabyte on modern systems.
Re:Mkfile(8) is severely syscall limited on OS X by mpweiher on Sunday February 12, 2017 @07:05AM (#13627986)
Neither did I! And in fact it took almost 2 days to question/overcome my assumptions.

"It's not what you don't know that kills you. It is what you know for sure that ain't true" -- Mark Twain

Re:Mkfile(8) is severely syscall limited on OS X by danieldk on Sunday February 12, 2017 @07:59AM (#13628131)
Interesting... You never dd data to a USB3 disk or SSD? The default buffer size for dd on OS X is also 512 bytes. I've been using `bs=1m` on OS X for ages.
Re:Mkfile(8) is severely syscall limited on OS X by sz4kerto on Sunday February 12, 2017 @08:06AM (#13628151)
Why are you using dd for USB disks?
Re:Mkfile(8) is severely syscall limited on OS X by danieldk on Sunday February 12, 2017 @08:07AM (#13628156)
Writing bootable installation images?
Re:Mkfile(8) is severely syscall limited on OS X by sz4kerto on Sunday February 12, 2017 @01:36PM (#13629749)
You should use cp, that's just better for this task.

"dd is not a disk writing tool" http://www.vidarholen.net/contents/blog/?p=479

Re:Mkfile(8) is severely syscall limited on OS X by chungy on Sunday February 12, 2017 @10:27AM (#13628750)
You could use cp.
Re:Mkfile(8) is severely syscall limited on OS X by jsjohnst on Sunday February 12, 2017 @11:39AM (#13629088)
cp is designed to copy data across file systems.

dd is designed to copy raw block data across devices.

Sure, you can abuse both for either purpose, but seems rather misguided to call it out explicitly. What did you intend to gain by doing it?

Re:Mkfile(8) is severely syscall limited on OS X by sz4kerto on Sunday February 12, 2017 @01:38PM (#13629759)
You don't know what cp is for. cp is exactly for copying data from a file to another file -- and under Unix, devices are represented as files. Using cp for writing disk ISOs to an USB drive is proper use, not abuse.

dd is for transform-and-copy, so you're abusing dd when writing disk ISOs. The fact that most of the Internet is wrong doesn't make you any less wrong.

Re:Mkfile(8) is severely syscall limited on OS X by chungy on Sunday February 12, 2017 @12:39PM (#13629433)
cp is designed to copy data from one file to another. Block devices are represented as files, and cp won't be speed limited by block size.

dd is also designed to copy data from one file to another, in blocks, with optional conversion of the data (eg, from EBCDIC to ASCII), and with optional offsets/limits to copy regions of a file.

dd is not designed to copy raw block data, it is merely a different file copy tool.

Re:Mkfile(8) is severely syscall limited on OS X by feld on Sunday February 12, 2017 @11:21AM (#13628997)
Explain?
Re:Mkfile(8) is severely syscall limited on OS X by fulafel on Sunday February 12, 2017 @08:41AM (#13628263)
It's just OS X being slow, Linux gets close to peak with a 16 kB buffer: http://blog.tdg5.com/tuning-dd-block-size/
Re:Mkfile(8) is severely syscall limited on OS X by alain94040 on Sunday February 12, 2017 @11:41AM (#13629094)
Reading from /dev/zero and writing out to RAM with a more optimal block size of 256K yields a throughput of 1,536 MB/s

Actually, in this article, Linux writing to RAM is slower than MacOS writing to SSD. Hard to conclude that "it's just OS X being slow". I suspect the hardware used is wildly different from the one in the original article.

Re:Mkfile(8) is severely syscall limited on OS X by mpweiher on Sunday February 12, 2017 @10:11AM (#13628658)
Maybe I am reading it wrong, but I am not getting the 16KB optimal buffer size from that article. The summary at the end is as follows:

'In the above example it can be seen that an input block size of 128K is optimal for my particular setup.'

However, the SSD seems to be significantly slower, so the relative overhead of the syscall is going to be respectively less:

'Reading from /dev/zero and writing out to a SSD with a more optimal block size of 256K yields a throughput of 280 MB/s.'

I was getting 250MB/s with the 512 byte buffer size, although the test in the article is doing two syscalls (one read(), one write()) whereas mkfile is just doing one.

At 256KB buffer size I was already getting around 1.8GB/second writing to SSD.

Re:Mkfile(8) is severely syscall limited on OS X by feld on Sunday February 12, 2017 @11:22AM (#13629001)
Or is it another case of Linux taking shortcuts and sacrificing safety for speed?
Re:Mkfile(8) is severely syscall limited on OS X by ivoras on Monday February 13, 2017 @09:45AM (#13635563)
Just saw the performance of this "consumer" device and remembered this thread: http://www.anandtech.com/show/11104/the-patriot-hellfire-m2-... . This SSD can write over a GB/s and read nearly 3 GB/s. We've come a long way since 32 KiB buffers were a sane choice.