News for hackers. Stuff that matters.
Re:What's wrong with 1975 programming by nkurz on Wednesday July 28, 2010 @09:05PM (#1556200)
> you'll still need to keep track of which data is on the disk and which data isn't

I'm not following this. As I see it, all you have to do is keep insure that no more than 3GB are mapped for your process at a given time. You still let the kernel decide what's faulted in and when things are written out.

Maybe my perspective is skewed: I'm viewing it from the perspective of real-time search, with large posting lists being frequently read and updated, and lazily written out to disk. Is there a different situation that is harder?

> In short, i386 is not dead. And suggesting that it cannot handle "gigabytes of data" is absurd

I didn't mean to imply it couldn't, or that it wasn't in common use. Rather I was wondering why I as a open source software author (http://wiki.apache.org/incubator/LucyProposal) would choose to make my life more complicated by targeting new software at a 32-bit platform at the expense of code simplicity.

Our current plan is to optimize for 64-bit systems, and allow 32-bit compatibility for smaller data sets (tens of GB) by sprinkling in mmap()/munmap() calls as necessary: http://www.mail-archive.com/[email protected]/msg00...

Edit to add: I'm not trying to claim I'm definitively right. Rather, my point is that I care about the answers to these questions in practice as well as in theory. I worry about taking the wrong approach.

There are no comments on this comment yet.