He is in less of a position to know whether you would be a good VC than you are. It's not about him - it's about what you want. People generally do well at things they enjoy or feel is valuable. You are a bright person. I would be more concerned as to whether he knows how to be a good VC:)