fs.promises.readFile is 40% slower than fs.readFile

Re:fs.promises.readFile is 40% slower than fs.readFile by orf on Wednesday March 03, 2021 @04:52PM (#26335172)

On a SSD reading files is fast. However read() is a blocking operation that could take forever - reading some /dev/ device that never returns any data, or a slower medium that takes hundreds of milliseconds per read. That’s a big no-no. So it gets dispatched to a dedicated blocking-io threadpool with limited workers which does the blocking operation without interrupting the event loop.

I’m guessing as disk speeds have increased the relative overhead of dispatching the task to a thread pool has also increased.

At least, that’s how I think it works based on some half-remembered blog post.

Edit: yep - http://docs.libuv.org/en/v1.x/design.html#file-i-o

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by beaconstudios on Wednesday March 03, 2021 @04:57PM (#26335247)

I can fully believe that there's some cleverness going on with node file I/O - but there should be no meaningful difference in performance between returning the result via a Promise, versus a callback.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by orf on Wednesday March 03, 2021 @05:19PM (#26335544)

It depends - I would assume the callback pattern is much easier to JIT than the promise pattern and requires less objects to be allocated, but you’re right. That’s a huge difference.

You’d need to benchmark another non-io bound function to find how much overhead comes from promises themselves.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by beaconstudios on Wednesday March 03, 2021 @05:22PM (#26335588)

Not enough to add 40% overhead to a 1mb file read, I can tell you that much.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by javajosh on Wednesday March 03, 2021 @07:43PM (#26337092)

It depends. If disks got so fast that function call overhead starts to dominate, then you might see an effect like this. But I have to imagine reading from an SSD is still 1000x slower (at least) than a function invocation in Javascript.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by beaconstudios on Wednesday March 03, 2021 @08:24PM (#26337408)

I don't see that basically ever being the case.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by SavantIdiot on Wednesday March 03, 2021 @10:47PM (#26338377)

The most recent comment is insightful:

"My guess is that the promise version is idling when IO capacity is available due to scheduling? Is it implemented in an event driven way internally, and if it is (for example using epoll) is the epoll event able to interrupt and cause the promise to continue to be evaluated or does it have to wait for the vm to get around to scheduling it again?"

It didn't occur to me that the async promise is rescheduled. That right there is a concern.

The OP also points out buffer concatenation, which is an odd choice. Reading the file into memory in one go should start with an stat and then a single buffer alloc. But that is probably naive.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by hinkley on Wednesday March 03, 2021 @05:27PM (#26335671)

Network filesystems, compressed filesystems, and of course the ever-dreaded on-access virus scanners on Windows that 'break' one open source project after another. Who knew that opening the same file 10 times is slow? Everybody. Everybody knew that.

Looking at you, svn team.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by TheCoelacanth on Wednesday March 03, 2021 @07:33PM (#26337012)

But these are both non-blocking APIs. readFileSync is the blocking version.

[ Parent | on HN ]

Re:fs.promises.readFile is 40% slower than fs.readFile by tynorf on Wednesday March 03, 2021 @07:45PM (#26337117)

I believe orf is talking about the read(2) syscall, which is blocking.

[ Parent | on HN ]