I started optimizing by making separate increment/render threads with a buffer and copy, and then doubled it to 4 threads. It's currently at 0xDD cycles but I don't think I'm going to take it farther:

http://i.imgur.com/sD5IAXP.png

The thread sync is pretty inefficient. It will need to incorporate some "tricky" techniques like what you've described to go faster.