News for hackers. Stuff that matters.
Re:Multicore version of bzip2 compression utility by wheels on Monday April 27, 2009 @04:48PM (#582078)
Tried it before and found it completely useless. The only place I cared about using parallel compression / decompression was on large files, and since it does everything in memory it dragged my system into a swapping hell every time that I used it.
Re:Multicore version of bzip2 compression utility by neilc on Monday April 27, 2009 @05:02PM (#582096)
You tried it, although it was just released today?

since it does everything in memory it dragged my system into a swapping hell

Well, it just keeps pending blocks in memory. It should be quite easy to bound the worst-case memory consumption (at some cost to parallelism), as they note in the article.

Re:Multicore version of bzip2 compression utility by evgen on Monday April 27, 2009 @05:16PM (#582131)
> You tried it, although it was just released today?

The version from clik was released today. Parallel bzip2 has been around for at least a couple of years. In fact, just doing a search for parallel bzip2 will lead you right to both an SMP/threaded version as well as one that uses MPI. I am guessing this is what was being referred to regarding the memory-consumption issues.

Re:Multicore version of bzip2 compression utility by wheels on Monday April 27, 2009 @05:11PM (#582115)
I assumed it was the same as this:

http://compression.ca/pbzip2/

Though looking again, it seems like it's a separate implementation.