Given the required front-end bandwidth, ADC bandwidth, and processing power, I don't think so. The closest thing you can do is perhaps get your hands on a FPGA board that has >10G serdes, and use the internal eye monitoring features to check the eye diagram, mask margin, etc.