What version are you using? Prior to 1.2 there was a nasty perf bug with :hlsearch that caused it to be sluggish with 1) very large files or 2) a good number of files opened. The bug is mostly fixed in 1.2 but there are still a couple of corner cases. Those are addressed in 1.3 which will be released sometime next week. A beta drop is available here
https://github.com/downloads/jaredpar/VsVim/VsVim-1.2-Beta2....
As for crashes I'd love to hear about any you found. Feel free to send me email (hacker news alias at microsoft.com) or post an issue on github.
Generally, I like it, but it can be a bit sluggish at times. The only plugin I have running is JSLint, so I suspect it's VSVim causing the problems (though it only generally bogs down when I leave VS2010 for a bit). The keybindings work as one would expect. As I mentioned in a different thread, I wish it could break out of the tabbed window convention of VS and use buffers.
ViEmu did feel a bit faster (and was less prone to crashing).