News for hackers. Stuff that matters.
Re:What's wrong with 1975 programming by cninja on Wednesday July 28, 2010 @04:38PM (#1555574)
What are the difficulties in using some sort of compression algorithm like LZJB on the disk's page file?

If, as you say, data can be that easily compressible, the IO speedups from reading the compressed data from disk and space savings would outweigh the CPU cycles necessary to do the decode, but my brief Google search for compressed page file implementations came up with nothing.

Re:What's wrong with 1975 programming by antirez on Wednesday July 28, 2010 @05:14PM (#1555701)
A linked list is hard to compress in memory as the overhead is the metadata: pointers, malloc overhead, ... but when written on disk can be represented as prefixed length strings.

With VM the live representation and serialization format are the same.

Re:What's wrong with 1975 programming by wrs on Thursday July 29, 2010 @05:12PM (#1559274)
Paul Wilson worked on compressed paging at the turn of the century [1] and tried to address this by introducing specialized compression algorithms for different kinds of pages (e.g., x86 instructions, pointer-filled data structures). Interesting stuff, but I don't think I've ever seen it commercialized, except in Newton OS where we used compressed code pages in binaries (read-only, not swap).

[1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.26.7...

Re:What's wrong with 1975 programming by signa11 on Wednesday July 28, 2010 @10:45PM (#1556436)
just curious: is it not possible to have some form of unrolled-linked-list to mitigate this effects somewhat ? or perhaps you have already tried it, and it doesn't really fit the bill.
Re:What's wrong with 1975 programming by nitrogen on Thursday July 29, 2010 @12:40AM (#1556653)
It kind of sounds like you're describing a sparse array.
Re:What's wrong with 1975 programming by signa11 on Thursday July 29, 2010 @01:09AM (#1556688)
> It kind of sounds like you're describing a sparse array.

hmm not really. this: [ http://en.wikipedia.org/wiki/Unrolled_linked_list ] is what i had in mind

Re:What's wrong with 1975 programming by nitrogen on Thursday July 29, 2010 @12:30AM (#1556639)
Here's something from Android: http://code.google.com/p/compcache/