For anything that just uses "standard" matrices/vectors I'd wager the best bet is to just use a C# wrapper around a native library such as MKL or something.

This seems like the choice when you want to have custom kernels run on your large arrays.