News for hackers. Stuff that matters.
Re:Computing file hashes with node.js by nakkiel on Tuesday November 01, 2011 @08:43AM (#3181701)
I ran the code, which now displays number of calls to update(), and made another test with the following code:

   7 fs.readFile(filename, function (err, data) {
   8     shasum.update(data);
   9     console.log(shasum.digest('hex') + '  ' + filename);
  10 });
Here are the results:

  $ dd if=/dev/urandom of=xxx.data bs=1000 count=200000
  $ time sha256sum xxx.data
  e774e4c46ab832ec09dbfd1a944044651560c3fdc3c5e2e8b46c2ea7d54f6649  xxx.data
  
  real	0m2.087s
  user	0m2.000s
  sys	0m0.064s
  $ time node xxx-1.js xxx.data
  e774e4c46ab832ec09dbfd1a944044651560c3fdc3c5e2e8b46c2ea7d54f  6649  xxx.data
  4883
  
  real	0m13.972s
  user	0m13.885s
  sys	0m0.264s
  $ time node xxx-2.js xxx.data
  e774e4c46ab832ec09dbfd1a944044651560c3fdc3c5e2e8b46c2ea7d54f6649  xxx.data
  
  real	0m14.043s
  user	0m13.433s
  sys	0m0.732s
Personally I never make multiple calls to update() but it seems not to be linked here. Oh, and check the docs, readFile() is async.

PS: timings of xxx-1.js and xxx-2.js are equivalent on several runs.

Re:Computing file hashes with node.js by SaltwaterC on Tuesday November 01, 2011 @09:50AM (#3181976)
The comment about blocking the event loop is all about hogging the CPU with the hash computation, stuff that's not light. See the "node.js is cancer" article for more details. fs.readFile(), although non-blocking, is worse since it actually buffers the data. Some files may be larger than the system memory. Kinda impossible in this case to buffer all the bytes. Even without multiple calls to update(), node is still dog slow.
Re:Computing file hashes with node.js by SaltwaterC on Tuesday November 01, 2011 @10:01AM (#3182019)
PS: the number of update() calls isn't the issue. Increasing the ReadStream buffer from the default 40 KiB to 1 MiB makes it 0.2s slower although there are 25X less update() calls.