The closest thing I'm aware of for doing that currently is the Phoronix Testing Suite; or whatever they're calling it now. It's most certainly not complete but it's the only thing I'm aware of that can do any kind of regression testing like that. In fact it's been worked on recently to make it really easy to do testing with git bisect on the kernel.