Re:Libtorrent experience - the poor state of async disk IO
by wmf on Friday October 26, 2012 @02:44PM
(#4703609)
Yeah, I would be more interested in hearing why libtorrent feels the need to implement their own disk cache. I'm sure they can do better, but by how much?
[ Parent
| on HN ]
Re:Libtorrent experience - the poor state of async disk IO
by jeffffff on Friday October 26, 2012 @05:23PM
(#4704330)
i actually very much doubt that they can do better. writing any sort of large LRU cache in a machine with swap turned on is a bad idea because a lot of your cache will get swapped out and then swapped back in unnecessarily when you try to use that memory for something else. mlock can be used to mitigate this effect but the default mlock limit is so low that it's useless in practice. another thing to consider is that using mmap gives you a big advantage over writing your own disk cache because the OS can take advantage of the paging hardware in the processor, which you can't do from userspace.
Re:Libtorrent experience - the poor state of async disk IO
by arvidn on Friday October 26, 2012 @10:25PM
(#4705116)
one reason why the bittorrent layer could make better decisions on what to flush is because it always need to hash pieces that are downloaded. When blocks are downloaded out of order, it doesn't make sense to flush unhashed blocks before ones that the the hash cursor has already passed. If an unhashed block is flushed, it will have to be read back from disk again, when the piece complete, which is very expensive.